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The  round  robin  method  is  considerably  simpler  than  other  known  strategies 
for  achieving  throughput  fairness.  The  fair  session  rates  are  not  explicitly 
computed,  and  the  only  overhead  communication  is  that  required  for  the  window 
acknowledgements.  The  main  drawback  is  that  large  windows  are  needed  to 
achieve  even  approximately  fair  throughputs  in  some  systems,  and  large  windows 
permit  large  cross-network  delays.  This  may  be  intolerable  for  some  users. 
However,  the  thesis  also  shows  that  if  a  session  elects  to  use  small  windows, 
its  packets  are  guaranteed  to  experience  small  cross-network  delays,  and  a 
certain  lower  bound  on  its  service  rate  is  still  guaranteed.  (This  service 
rate  determines  the  maximum  session  throughput  rate  that  can  be  supported  and 
also  roughly  determines,  for  a  given  throughput  rate,  the  delay  of  packets 
waiting  to  be  admitted  to  the  network.)  These  guarantees  for  sessions  with 
small  windows  apply  even  if  other  sessions  in  the  network  are  using  larger 
windows.  Thus  the  round  robin  method  seems  to  be  well  suited  to  integrated 
services  networks.  Delay-sensitive  sessions  can  use  small  windows  to  meet 
their  needs,  and  ^he  remaining  transmission  capacity  can  be  fairly  divided 
among  the  other  sessions  by  assigning  them  large  windows.  Moreover,  a  session 
with  throughput  and/or  delay  requirements  too  stringent  to  be  met  simply  by 
proper  window  sizing  could  be  given  priority  service  by  being  visited  more  than 
once  in  each  polling  cycle. 
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ABSTRACT 


This  thesis  studies  a  simple  strategy  for  fairly  allocating  link  capacity  in  a 
point-to-point  packet  network  with  virtual  circuit  routing.  Each  link  offers  its 
packet  transmission  slots  to  its  user  sessions  by  polling  them  in  round  robin 
order.  In  addition,  link-by-link  window  flow  control  is  used  to  prevent 
excessive  packet  queues  at  the  network  nodes.  As  the  window  size  increases, 
the  session  throughput  rates  are  shown  to  approach  limits  that  are  perfectly 
fair  in  the  max-min  sense.  That  is,  the  smallest  session  rate  in  the  network  is 
as  large  as  possible  and,  subject  to  that  constraint,  the  second-smallest  session 
rate  is  as  large  as  possible,  etc.  If  each  session  has  evenly  spaced  packet 
arrivals  or  has  such  heavy  demand  that  packets  are  always  waiting  to  enter  the 
network,  then  a  finite  window  size  suffices  to  produce  perfectly  fair  throughput 
rates.  (These  properties  do  not  hold  if  first-come-first-served  scheduling  is 
used  instead  of  round  robin.) 


The  round  robin  method  is  considerably  simpler  than  other  known 
strategies  for  achieving  throughput  fairness.  The  fair  session  rates  are  not 
explicitly  computed,  and  the  only  overhead  communication  is  that  required  for 
the  window  acknowledgments.  The  main  drawback  is  that  large  windows  are 
needed  to  achieve  even  approximately  fair  throughputs  in  some  systems,  and 
large  windows  permit  large  cross-network  delays.  This  may  be  intolerable  for 


some  users.  However,  the  thesis  also  shows  that  if  a  session  elects  to  use  small 
windows,  its  packets  are  guaranteed  to  experience  small  cross-network  delays, 
and  a  certain  lower  bound  on  its  service  rate  is  still  guaranteed.  (This  service 
rate  determines  the  maximum  session  throughput  rate  that  can  be  supported 
and  also  roughly  determines,  for  a  given  throughput  rate,  the  delay  of  packets 
waiting  to  be  admitted  to  the  network.)  These  guarantees  for  sessions  with 
small  windows  apply  even  if  other  sessions  in  the  network  are  using  larger 
windows.  Thus  the  round  robin  method  seems  to  be  well  suited  to  integrated 
services  networks.  Delay-sensitive  sessions  can  use  small  windows  to  meet  their 
needs,  and  the  remaining  transmission  capacity  can  be  fairly  divided  among  the 
other  sessions  by  assigning  them  large  windows.  Moreover,  a  session  with 
throughput  and/or  delay  requirements  too  stringent  to  be  met  simply  by 
proper  window  sizing  could  be  given  priority  service  by  being  visited  more  than 
once  in  each  polling  cycle. 
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1.  INTRODUCTION 


1.1  Problem  Statement  and  Background 

Consider  a  data  communication  network  consisting  of  store-and-forward 
switches  (nodes)  joined  by  point-to-point  communication  channels  (links). 
Each  network  user  (session)  is  assigned  a  fixed  path  (virtual  circuit)  through 
the  network,  and  data  for  the  session  are  sent  in  manageable  parcels  ( packets ) 
along  this  path.  In  such  a  network,  occasional  surges  in  user  demand  can 
overload  network  links,  causing  packet  queues  to  build  up  in  network  nodes. 
These  queues  may  eventually  overflow  the  nodes’  storage  space,  or  the  delay  of 
acknowledgments  may  cause  transmitters  to  assume  that  data  were  lost.  These 
problems  result  in  wasteful  retransmissions  that  effectively  reduce  the  capacity 
of  the  network.  Flow  control  procedures  attempt  to  prevent  or  alleviate  this 
degradation  by  regulating  the  appropriate  traffic  sources.  Gerla  and  Kleinrock 
[10]  discuss  many  of  the  flow  control  techniques  that  have  been  proposed  in  the 
literature. 

One  such  scheme  is  the  window  method  [10].  This  technique  limits  the 
number  of  packets  for  each  session  that  have  been  transmitted  but  for  which 
acknowledgments  have  not  yet  been  received.  The  maximum  permissible 
number  of  outstanding  packets  is  called  the  window  size.  In  the  end-to-end 
method,  a  single  window  is  applied  to  all  of  a  session’s  traffic,  and  the  session’s 
destination  node  sends  an  acknowledgment  to  the  origin  node  whenever  a 
packet  is  claimed  by  the  session’s  sink.  In  the  link-by-link  or  node-by-node 
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method,  the  session  has  a  separate  window  for  its  traffic  over  each  link,  and 
whenever  a  packet  is  transmitted  from  a  node,  that  node  sends  an 
acknowledgment  to  the  packet’s  preceding  node.  The  link-by-link  method  is 
equivalent  to  each  session  having  a  dedicated  storage  area  ( buffer )  at  each  node 
in  its  path;  the  buffer  capacity  equals  the  window  size.  The  window  method  is 
described  here  because  it  is  a  component  of  several  more  elaborate  strategies  to 
be  discussed  later. 

It  would  be  desirable  for  flow  control  procedures  to  regulate  network  inputs 
so  as  to  grant  each  session  a  fair  throughput  rate.  Gerla  and  Kleinrock  [10] 
explain  that  many  proposed  flow  control  methods  are  unfair.  Several  studies 
have  addressed  the  issue  of  throughput  fairness,  however,  and  these  will  now 
be  briefly  discussed. 

The  problem  of  achieving  fair  throughput  rates  can  be  broken  into  three 
parts.  First  the  fairness  objective  must  be  formulated  precisely.  Then  the  fair 
session  rates  must  be  determined.  Finally,  these  rates  must  be  enforced. 
Hayden  [13:  Chapter  3],  Regnier  [23],  Golestaani  and  Gallager  [12,  8],  Gerla  and 
Staskauskas  [11:  Section  5.2],  Thaker  and  Cain  [26],  Ibe  [14],  Gafni  [5:  Sections 
4  and  6.2],  Sauve,  Wong  and  Field  [24,  25],  and  Bharath-Kumar  and  Jaffe  [l, 
IS]  have  objectives  of  roughly  the  same  form.  They  seek  to  maximize  a  sum  of 
functions,  one  for  each  session.  For  Hayden,  each  term  gives  the  satisfaction  of 
a  session  as  a  function  of  its  throughput  rate.  Regnier  considers  both  the 
throughput  rate  of  a  session  and  its  average  packet  delay.  Golestaani  and 


Gallager,  Gerla  and  Staskauskas,  Thaker  and  Cain,  Ibe,  and  Gafni  express  a 
session’s  happiness  as  a  function  of  only  its  throughput  rate,  but  extra  terms 
are  added  to  penalize  high  link  delays.  Sauve,  Wong  and  Field  use  a 
performance  measure  that  depends  on  the  ratio  of  a  session’s  throughput  rate 
to  the  total  network  throughput  rate.  Bharath-Kumar  and  Jaffe  measure  a 
session’s  success  by  its  power  (i.e.,  throughput  divided  by  delay)  or  the 
logarithm  of  its  power.  Another  fairness  approach,  called  max-min  flow  control 
or  bottleneck  flow  control,  is  used  in  various  forms  by  Bially,  Gold,  and  Seneff 
[2],  Jaffe  [16,  17],  Gerla  and  Staskauskas  [11:  Section  3],  Hayden  [13:  Chapter 
4],  Gafni  [5:  Chapter  3],  Gafni  and  Bertsekas  [6],  Oshinsky  [22],  and  Mosely 
[20].  Only  the  simplest  version  of  this  objective,  viz.,  Hayden’s,  will  be  defined 
here.  To  satisfy  the  max-min  flow  criterion,  the  smallest  session  rate  in  the 
network  must  be  as  large  as  possible.  Subject  to  this  constraint,  the  second- 
smallest  session  rate  must  be  as  large  as  possible,  etc.  Given  a  network  with  its 
link  capacities  and  a  set  of  sessions  with  their  routes  and  their  maximum 
possible  transmission  rates,  there  is  a  unique  set  of  session  rates  that  satisfies 
the  max-min  conditions.  The  max-min  flow  criterion  will  be  taken  as  the 
definition  of  throughput  fairness  in  this  thesis.  Most  of  the  studies  mentioned 
in  this  paragraph  also  develop  algorithms  for  computing  session  rates  that  are 
fair  according  to  the  various  criteria.  Many  of  these  algorithms  are  meant  to 
be  implemented  in  a  distributed  manner.  (In  an  interesting  twist,  Jaffe  and 
Bharath-Kumar  [18,  l]  argue  that  power  is  not  a  suitable  objective  because  it  is 


neither  convex  nor  decentralizable;  algorithms  to  optimize  power  would  be 
impractical.) 


Beyond  the  issues  of  defining  and  computing  fair  rates  is  the  problem  of 
enforcing  them.  Several  methods  have  been  suggested  in  the  literature. 
Golestaani  and  Gallager  [12,  8],  Thaker  and  Cain  [26],  Ibe  [14:  Section  4.7], 
Gafni  [5:  Chapter  5],  and  Gerla  and  Staskauskas  [11:  Section  5.2]  use  window 
flow  control  and  adjust  the  sessions’  window  sizes  to  achieve  the  desired  rates. 
Bially,  Gold  and  Seneff  [2],  Hayden  [13:  Chapter  5],  Mosely  [20],  Ibe  [14:  Section 
4.2],  Gafni  [5:  Chapter  3],  and  Gafni  and  Bertsekas  [6]  consider  a  session  input 
control  that  produces  packet  lengths  proportional  to  the  session’s  assigned 
rate.  The  time  between  packet  admissions  is  constant.  This  model  is 
particularly  meaningful  for  packetized  voice  traffic:  it  represents  the  output  of 
a  variable  rate  vocoder  [2].  Oshinsky  [22]  takes  the  opposite  approach,  called 
metering.  Time  is  divided  into  control  intervals  whose  length  is  inversely 
proportional  to  a  session’s  target  rate.  The  session  is  permitted  to  inject  some 
fixed  quantity  of  data  into  the  network  during  one  control  interval.  A  fourth 
approach,  taken  by  Mukherji  [21]  and  Sauve,  Wong  and  Field  [24,  25]  is  to 
schedule  the  use  of  the  links  among  the  various  sessions.  These  studies  assume 
that  window  flow  control  is  also  used,  but  it  is  primarily  the  schedule 
parameters  rather  than  the  window  sizes  that  are  adjusted  to  achieve  the 


desired  session  rates. 


This  thesis  studies  the  following  new  method  proposed  by  Gallager  [7,  9]  for 
max-min  fair  flow  control.  Each  link  offers  its  packet  transmission  slots  to  its 
users  by  polling  them  in  round  robin  order.  If  a  session  is  offered  a  chance  to 
use  a  link  slot  but  has  no  packets  ready,  then  that  same  slot  is  offered  to  the 
next  session,  and  perhaps  the  next,  etc.,  until  a  ready  session  is  found.  In  each 
pass  of  a  link’s  round  robin,  a  session  may  transmit  only  one  packet.  The 
round  robin  schedulers  for  the  various  links  are  uncoordinated.  In  order  to 
prevent  excessive  queues  at  the  network  nodes,  window  flow  control  is  also 
employed.  The  principal  contribution  of  this  thesis  is  a  proof  that  round  robin 
scheduling  with  windows  can  indeed  be  used  to  achieve  max-min  throughput 
fairness. 

The  main  advantage  of  the  round  robin  method  is  its  simplicity.  The 
desired  rates  are  never  explicitly  computed,  as  they  are  for  other  fair  flow 
control  schemes.  The  only  overhead  communication  is  that  required  for  the 
window  acknowledgments.  The  window  sizes  need  not  be  adjusted  as  network 
conditions  change.  An  obvious  price  paid  for  this  simplicity  is  a  lack  of 
flexibility.  The  strategy  is  coupled  to  the  max-min  criterion  and  probably 
cannot  be  adapted  to  fairness  objectives  substantially  different  from  max-min. 
(Session  priorities  can  be  implemented,  however,  by  allowing  some  sessions  to 
transmit  more  than  one  packet  over  a  link  in  each  polling  cycle.)  Another 
drawback  is  that  large  windows  are  needed  to  ensure  throughput  fairness  for 
some  networks,  and  large  windows  permit  large  cross-network  delays. 


However,  the  thesis  also  proves  certain  throughput  guarantees  that  apply  even 
for  sessions  using  small  windows. 


1.2  Thesis  Overview 

The  system  model  assumed  throughout  this  thesis  is  described  in  Chapter  2. 
The  network  is  composed  of  nodes  with  ample  storage  connected  by  point-to- 
point,  error-free  links  with  negligible  propagation  delays  and  equal  capacities. 
Uniform  packet  lengths  are  also  assumed,  so  that  the  time  required  to  transmit 
one  packet  over  one  link  is  the  same  for  all  packets  and  all  links.  This 
fundamental  time  unit  is  called  a  slot.  The  network  supports  a  fixed  set  of  5 
sessions.  Virtual  circuit  routing  is  used,  with  no  more  than  H  links  in  the  path 
of  any  session  and  no  more  than  N  sessions  sharing  any  single  link.  Window 
flow  control  operates  on  a  link-by-link  basis,  with  window  sizes  of  at  most  W 
packets.  Several  link  scheduling  disciplines  are  modeled,  including  round 
robin,  first-come-first-served,  and  a  generalization  of  these  called  bounded 
delay  scheduling.  A  link  scheduling  discipline  is  said  to  have  bounded  delay  if 
any  session  with  an  open  window  is  guaranteed  at  least  a  certain  service 
frequency.  A  variety  of  deterministic  and  random  models  are  considered  for 
packet  arrivals,  but  packets  are  assumed  to  depart  as  soon  as  they  reach  their 
destination  nodes. 

Chapter  4  studies  the  session  throughputs  in  systems  using  round  robin  link 
scheduling  and  large  windows.  There  are  two  major  results,  one  assuming 
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bursty  packet  arrivals  and  one  for  smooth  demand.  Theorem  4  asserts  that,  for 
Bernoulli  packet  arrivals,  the  session  throughput  rates  approach  the  max-min 
fair  rates  as  the  window  sizes  increase  (provided  they  all  increase  at  the  same 
rate).  If  each  session  has  evenly  spaced  packet  arrivals  or  has  such  heavy 
demand  that  packets  are  always  waiting  to  enter  the  network,  then  the  traffic 
admitted  to  the  network  will  be  rather  smooth.  For  this  model,  Corollary  1 
claims  that  the  long-term  average  throughputs  equal  the  fair  rates  if  the 
windows  are  at  least  a  certain  size  ky  .  Chapter  4  includes  several  other 
results  for  this  smooth  demand  model.  In  Theorem  2,  the  throughput  of  a 
session  during  any  finite  interval  is  shown  to  be  within  a  constant  Jb2  of  the 
max-min  fair  number  of  packets  (regardless  of  the  length  of  the  interval).  This 
constant  k2  is  proportional  to  the  window  size,  because  the  system  cannot 
reach  a  steady  state  until  the  buffers  upstream  of  a  session’s  most  congested 
link  fill  up  with  packets  and  the  downstream  buffers  drain.  According  to 
Theorem  3,  a  steady  state  is  eventually  reached;  thereafter,  the  unfairness  of  a 
session’s  throughput  over  any  interval  is  less  than  another  constant  k3  <  k2  . 
This  steady-state  unfairness  bound  kz  is  less  than  the  minimum  acceptable 
window  size  k{  ,  but  is  independent  of  the  actual  window  sizes.  This  suggests 
that  using  windows  larger  than  the  minimum  required  value  kx  may  cause  a 
longer  transient,  but  probably  does  not  affect  the  throughput  performance  of 
the  system  in  steady  state.  Corollaries  3  and  4  show  that,  in  steady  state, 
buffers  upstream  of  a  session’s  most  congested  link  are  never  empty,  while 


buffers  downstream  are  never  full,  so  that  the  session  accepts  every  chance 
offered  to  it  by  the  round  robin  scheduler  at  its  most  congested  link.  (Of 
course,  the  term  "most  congested  link"  must  be  precisely  defined,  and  if  a 
session  has  more  than  one  such  link,  the  results  are  more  complicated.) 
Unfortunately,  the  minimum  window  size  kx  for  which  Corollary  1  proves 
perfect  fairness  of  the  long-term  average  throughputs  is  absurdly  large  and 
grows  exponentially  in  S.  Moreover,  recall  that  for  the  bursty  demand  result, 
no  finite  window  size  was  sufficient.  One  wonders  how  large  the  windows  must 
be  to  achieve  at  least  approximate  fairness  in  practice.  In  Example  3,  the 
session  throughput  rates  are  quite  unfair  unless  very  large  windows  are  used. 
The  example,  however,  is  somewhat  contrived,  leaving  open  the  question  of 
performance  for  "typical"  systems.  Also  included  are  Examples  6  and  7,  which 
use  first-come-first-served  link  scheduling  instead  of  round  robin  scheduling. 
In  these  examples,  the  session  throughput  rates  are  unfair  even  if  large 
windows  are  used.  The  throughput  rates  are  shown  to  be  very  sensitive  to  the 
relative  window  sizes  of  competing  sessions  and  to  the  initial  conditions  of  the 
system,  even  if  the  windows  are  large.  This  contrasts  sharply  with  the  round 
robin  results. 

Chapter  4  seeks  max-min  fair  throughputs,  asking  how  large  the  windows 
must  be.  Chapter  5  takes  a  different  approach.  It  assumes  only  that  the 
window  sizes  are  at  least  two  packets  and  asks  how  unfair  the  throughputs  can 
be.  Bounded-delay  link  scheduling  is  assumed,  and  bursty  demand  is 


permitted.  The  chapter  presents  two  theorems  that  differ  in  their  assumptions 
about  how  many  packets  can  be  stored  at  a  session’s  source.  Theorem  6  shows 
that  if  round  robin  scheduling  is  used,  then  the  throughput  rate  of  a  session 
with  demand  rate  X  packets/slot  and  unlimited  source  buffering  is  at  least 
min[X  ,  1  / N\  packets/slot.  This  throughput  guarantee  is  within  a  factor  of  N 
of  the  max-min  fair  rate.  The  analogous  bound  for  first-come-first-served 
scheduling  is  roughly  min[X  ,  l/(./V*  W)\  packets/slot.  This  throughput 
guarantee  is  worse  than  the  round  robin  guarantee  by  roughly  a  factor  of  W  if 
X  is  large.  Theorem  5  gives  other  throughput  guarantees  for  sessions  whose 
source  buffers  are  finite  but  hold  at  least  two  packets.  These  guarantees  are 
too  complicated  to  describe  here,  but  if  the  session’s  demand  rate  X  >  1 
packet/slot,  then  the  guarantees  match  those  given  above  for  unlimited  source 
buffering.  Round  robin  Example  1  (Section  3.2)  and  first-come-first-served 
Example  2  (Section  3.3)  show  a  session  x  with  demand  rate  X  =  1  packet/slot 
whose  throughput  rate  (for  either  source  buffering  assumption)  matches  the 
theoretical  lower  bound,  f  Example  3  (Section  4.4.2)  shows  that,  with  round 
robin  scheduling,  the  throughput  rate  of  a  session  with  heavy  demand  and 

f  In  the  first-come-first-served  example,  however,  session  x  must  have  at  least 
one  window  of  size  two  packets,  while  the  other  sessions  may  have  arbitrarily 
large  windows.  I  suspect  that  if  all  sessions  are  required  to  have  the  sum*' 
window  size  W  and  \V  is  arbitrary  (except  that  H'  >  2  packets),  then  the 
worst  case  throughput  for  first-come-first-served  systems  is  roughly  half  that 
of  round  robin  systems.  Moreover,  a  different  implementation  of  first-come- 
first-served  scheduling  than  that  assumed  here  might  perform  better. 


small  windows  really  can  be  unfair  by  a  factor  proportional  to  N.  For  first- 
come-first-served  scheduling,  Example  7  (Section  4.6.2)  shows  an  unfairness 
factor  proportional  to  N  and  Example  2  (Section  3.3)  shows  an  unfairness 
factor  roughly  equal  to  W. 

While  throughput  fairness  is  the  primary  focus  of  this  study,  cross-network 
delay  is  also  of  interest.  Consider  a  session  x  whose  window  size  w  is  at  least 
two  packets.  Of  course,  Little’s  formula  [19,  4]  can  take  a  given  lower  bound  of 
R  packets/slot  for  x’s  throughput  rate  (such  as  the  bounds  given  in  Chapters  4 
and  5)  and  generate  an  upper  bound  of  roughly  w'H/R  slots  for  the  average 
cross-network  delay  of  x’s  packets.  Theorem  1  of  Chapter  3,  however,  derives 
an  upper  bound  on  delay  that  applies  to  each  packet  and  is  tighter  in  some 
cases  than  the  bound  from  Little’s  formula.  The  theorem  assumes  bounded- 
delay  link  scheduling.  For  round  robin  scheduling,  the  upper  bound  on  x’s 
cross-network  delay  is  roughly  w’H’N  slots.  The  analogous  bound  for  first- 
come-first-served  scheduling  is  roughly  w’H’N'W  slots  —  worse  by  a  factor  of 
W.  Round  robin  Example  1  and  first-come-first-served  Example  2  show 
packets  whose  cross-network  delays  come  close  to  the  theoretical  upper 
bounds,  f 

t  In  the  first-come-first-served  example,  however,  session  x  must  have  at  least 
one  window  of  size  two  packets,  while  the  other  sessions  may  have  arbitrarily 
large  windows.  I  suspect  that  if  all  sessions  are  required  to  have  the  same 
window  size  W  and  IF  is  arbitrary  (except  that  \V  >  2  packets),  then  the 
worst  case  delay  for  first-come-first-served  systems  is  roughly  twice  that  of 
round  robin  systems.  Moreover,  a  different  implementation  of  first-come- 
first-served  scheduling  than  that  assumed  here  might  perform  better. 
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Chapter  6  discusses  the  practical  implications  of  this  research,  more 
thoroughly  compares  the  round  robin  method  with  the  fair  flow  control 
strategies  mentioned  in  Section  1.1,  and  offers  suggestions  for  future  study.  A 
glossary  of  notation  appears  at  the  end  of  the  document. 


2.  SYSTEM  MODEL 


This  chapter  presents  a  system  model  to  be  studied  in  Chapters  3,  4  and  5. 
The  model  features  uniform  link  capacities,  uniform  packet  lengths,  a  fixed  set 
of  users,  virtual  circuit  routing,  and  link-by-link  window  flow  control.  Several 
link  scheduling  disciplines  are  modeled,  including  round  robin,  first-come-first- 
served,  and  a  generalization  of  these  called  "bounded  delay"  scheduling.  A 
variety  of  deterministic  and  random  models  are  considered  for  packet  arrivals, 
but  packets  are  assumed  to  depart  as  soon  as  they  reach  their  destination 
nodes. 

2.1  Nodes,  Links,  Packets,  and  Time 

The  network  consists  of  store-and-forward  nodes  joined  by  point-to-point 
links.  A  link  allows  communication  in  only  one  direction.  If  two  nodes  are 
connected  by  link(s)  in  one  direction,  then  they  must  be  connected  by  at  least 
one  link  in  the  reverse  direction  so  that  flow  control  acknowledgments  can  be 
returned.  Except  for  this  restriction,  two  nodes  may  have  any  number  of  links 
connecting  them. 

Links  are  perfectly  reliable,  i.e.,  they  never  lose  or  corrupt  data,  and  they 
never  fail.  Nodes,  too,  are  perfectly  reliable,  and  the  storage  capacity  of  each 

node  is  infinite. 

Data  are  transmitted  through  the  network  in  packets  of  equal  length.  A 


time  interval  during  which  one  packet  is  transmitted  over  a  link  is  called  a 
time  slot  for  that  link.  All  links  have  the  same  capacity;  hence  all  time  slots 
have  the  same  length.  A  packet  experiences  no  processing  delay  at  a  node, 
other  than  a  possible  queuing  delay  as  it  waits  for  transmission.  A  packet 
experiences  no  propagation  delay  on  a  link.  The  transmission  time  slots  of  all 
links  are  synchronized;  hence  the  entire  network  operates  with  slotted  time. 
Although  the  operation  of  the  system  during  a  time  slot  will  occasionally  be 
discussed,  a  discrete-time  system  model  will  normally  be  used  in  which  the  tth 
discrete-time  instant,  called  time  t,  refers  to  the  end  of  the  tth  time  slot.  The 
model  begins  at  time  0. 

It  is  often  necessary  to  refer  to  intervals  longer  than  one  slot.  Let  [5,  t\ 
denote  the  interval  from  the  beginning  of  slot  s  to  the  end  of  slot  t  ;  if 
s  >  t  ,  then  [s,  t]  is  null.  Define  (s,  tj  ,  [s,  t)  ,  and  (s,  t)  as  follows: 


(1) 

(s,  t]  = 

[5+1,  t] 

(2) 

[«»  t)  - 

[s,  t-  1] 

(3) 

(s,  t)  = 

(s+1,  f-l] 

2.2  Sessions,  Routes,  Throughputs,  and  Delays 

The  network  supports  one-way  communication  activities  called  sessions. 
Each  session  is  assigned  a  path  (i.e.,  a  sequence  of  appropriately  directed  links) 
through  the  network  from  its  origin  node  to  its  destination  node:  data  packets 
for  the  session  are  transmitted  along  this  path.  Several  sessions  may  have  the 


same  origin  and  destination  nodes.  The  set  of  sessions  using  the  network  is 
fixed.  Let  5  denote  the  number  of  sessions  using  the  network,  and  let  N'(l) 
denote  the  number  of  sessions  using  link  l. 

While  all  links  have  global  identifiers,  it  is  often  convenient  to  index  links 
in  the  order  in  which  some  session  uses  them.  Therefore,  let  H(x )  denote  the 
number  of  links  in  the  path  of  session  x,  and  let  H  be  the  maximum  number  of 
links  in  the  path  of  any  session: 

(4)  H  —  max  H(x ) 

X 

For  each  session  x  and  for  h  =  1,  2,  ...  ,  H(x)  ,  define  hop  h  as  the  htk  link 
in  the  path  of  x,  including  the  related  functions  in  the  node  at  the  input  end 
of  that  link.  To  streamline  the  analysis,  packet  arrivals  and  departures  are 
modeled  as  transmissions  over  dummy  hops.  The  session’s  source  is  hop  0. 
The  session’s  sink  is  hop  H(x)+1  .  The  source  and  sink  are  considered  to  be 
hops  but  not  links.  For  h  —  1,  2,  ...  ,  H(x )  ,  let  N(x,  h)  denote  the  number 
of  sessions  using  hop  h  of  session  z,  including  x  itself.  Let  .V(z)  be  the 
maximum  number  of  sessions  using  any  link  in  the  path  of  x  : 

(5)  jV(j)  =  max  .V(x,  h) 

Let  A  be  the  maximum  number  of  sessions  sharing  any  link  in  the  network: 


(6) 


N  =  max  N(x)  =  max  N\l) 
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Packets  waiting  to  be  transmitted  over  hop  h  of  session  x, 
0  <  h  <  H(x)+1  ,  are  said  to  be  in  buffer  h.  The  number  of  packets  in  buffer 
h  of  session  x  at  time  t  is  called  the  buffer  level  B(x,  h,  t)  .  For  convenience, 
packet  arrivals  for  session  x  are  modeled  as  occasional  services  at  hop  0.  Buffer 
0  is  assumed  to  contain  an  infinite  number  of  packets  at  time  0.  In  each  time 
slot,  the  session’s  source  (hop  0)  transfers  either  one  packet  or  no  packets  from 
buffer  0  to  buffer  1.  Therefore,  for  all  times  t  >  0  , 

(7)  J3(x,  0,  t)  =  oo 

The  only  significance  of  (7)  is  that  buffer  0  is  never  empty.  The  initial  levels 
B(x,  h,  0)  of  the  buffers  h  =  1,  2,  ...  ,  H(x)  are  assumed  to  be  finite  but  are 
not  necessarily  zero.f  It  is  assumed  that  the  initial  level  B(x ,  #(x)+l,  0)  of 
buffer  H(x)+l  is  at  most  one,  and  that  the  session’s  sink  is  very  fast.  J  That 
is,  whenever  B{x,  H{x)+ 1,  t—  l)  >  0  ,  hop  /f(x)+l  removes  one  packet  from 
buffer  H{x)+1  during  slot  t.  Therefore,  for  all  times  t  >  0  , 


f  Since  the  initial  buffer  levels  can  be  positive,  the  assumption  of  a  fixed  set  of 
sessions  is  less  restrictive  than  it  appears.  The  approach  here  is  equivalent  to 
studying  a  more  realistic  model,  one  with  a  dynamic  set  of  sessions,  during 
an  interval  in  which  no  existing  sessions  are  terminated  and  no  new  sessions 
are  initiated. 

t  Fast  session  sinks  have  been  assumed  for  simplicity.  This  assumption  could 
be  relaxed  by  using  a  sink  model  similar  to  the  source  model. 
Straightforward  modifications  would  adapt  Chapter  4  to  such  a  model.  It  is 
not  clear  whether  Chapters  3  and  5  could  also  be  generalized. 
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(8)  5(x,/f(x)+l,  t)  <  1 

The  throughput  (measured  in  packets)  of  session  x  over  hop  h, 
0  <  h  <  tf(x)+l  ,  during  interval  (s,  f]  is  denoted  by  P{x ,  h,  s,  t)  .  For 
s  >  t  ,  P(x,  h ,  s,  t)  is  defined  to  be  zero.  Note  that,  for  s  <  t  <  u  , 

(9)  P(x,  h,  s,  u )  =  P(x,  h,  s,  t)  +  P(x,  h,  t,  u) 

If  link  l  is  hop  h  for  session  x,  then  P'(x,  l,  s,  t)  is  defined  to  equal 
P(x,  h,  s,  t)  .  There  is  a  simple  and  obvious  relationship  between  buffer  levels 
and  throughputs.  For  any  session  x,  any  buffer  h  of  x  in  the  range 
1  <  h  <  H(x)+ 1  ,  and  any  times  s  and  t  such  that  0  <  s  <  t  , 

(10)  B(x,  h,  t)  —  B(x ,  h,  s)  +  P(x,  A  —  1,  s,  £)  —  P{x,  h,  s,  t) 

The  long-term  average  throughput  Ra(x)  oi  a  session  x  is  defined  as  follows: 

(11)  Ra(x)  =  lim  P(X'  H&'  0>  ^ 

t—  oo  t 

This  limit  may  not  exist  for  systems  with  irregular  demand. 

Sequence  numbers  are  assigned  to  the  packets  of  each  session  x.  If  buffers 
h  =  1,  2,  ...  ,  H(x )  contain  any  packets  at  time  0,  then  the  one  farthest 
downstream  is  called  packet  1;  if  these  buffers  are  initially  empty,  then  the 
first  packet  to  arrive  at  buffer  1  after  time  0  is  called  packet  1.  The  packet 
following  packet  1  is  called  packet  2,  etc.  (If  buffer  H(x)+l  initially  contains 
a  packet,  that  packet  gets  no  sequence  number.) 


Let  T(x,  A,  p)  denote  the  time  slot  during  which  packet  p  >  1  of  session 
x  crosses  hop  A,  0  <  A  <  H(x)+l  .  For  convenience,  let  T(x,  A,  p)  be 
defined  for  all  integers  p,  with  T(x,  A,  p)  =  0  if  p  <  1  or  if  packet  p  is 
farther  downstream  than  buffer  A  at  time  0.  If  a  packet  p  >  1  gets  stuck  in 
a  buffer  A,  0  <  A  <  #(x)  ,  and  never  advances,  take  T(x,  h,  p)  —  oo  for 
A  <  A  <  /f(x)+l  .  Consider  the  following  claim: 

(12)  T(x, /f(x)+l,  p)  <  T(x, /f(x),  p  + 1)  for  all  integers  p 

For  p  <  1  ,  (12)  holds  because  its  left-hand  side  equals  zero.  For  p  >  1  ,  (12) 
holds  because  the  assumptions  about  buffer  /f(x)-f  1  and  hop  H(x)-r  1  imply 
that 

T(x,£T(x)+l,p)  -  T(xtH(x),p)  +  1  <  T(x,ff(x),p  +  1) 

For  each  packet  p  >  1  of  session  x  that  gets  beyond  buffer  1  (i.e.,  for 
which  T(x,  1,  p)  <  oo),  define  the  cross-network  delay  =(x,  p)  as  follows: 

(13)  =(x,  p)  -  T(x,H(x),p)  —  T(x,  1,  p)  +1 

This  measures  the  transmission  delays  across  hops  1  through  #(x)  ,  inclusive, 
plus  the  queuing  delays  in  buffers  2  through  H(x)  ,  inclusive. 

2.3  Link-by-Link  Window  Flow  Control 

The  capacity  (in  packets)  of  a  buffer  A  of  a  session  x,  0  <  A  <  //(x)+  1  , 
is  called  the  window  size  associated  with  that  buffer  and  is  denoted  by 
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W{x,  h)  .  For  each  session  x,  buffer  0  has  infinite  capacity: 

(14)  =  oo 

The  capacity  VK(x,  l)  of  buffer  1  may  be  either  finite  or  infinite;  this  permits 
a  wider  variety  of  packet  arrival  models.  The  buffers  h  in  the  range 
2  <  h  <  H(x)  must  have  finite  capacity.  The  capacity  of  buffer  H(x)+ 1  is 
assumed  to  be  at  least  two  but  finite: 


(15)  2  <  W{x,H(x)+ 1)  <  oo 
It  follows  from  (15)  and  (8)  that 

(16)  B{x,  H{x)+1,  t)  <  W{x,H{x)+ 1) 

for  all  times  t  >  0  .  In  other  words,  buffer  H(x)+ 1  is  never  full.  Buffers  2 
through  H(x)+ 1  are  required  to  be  finite  for  two  reasons  —  to  bound  the 
cross-network  delay,  and  to  keep  individual  sessions  from  consuming  grossly 
unfair  amounts  of  link  capacity.  Each  result  in  Chapters  3,  4  and  5  requires  its 
own  additional  assumptions  about  the  window  sizes.  The  maximum  window 
size  for  any  session  x  at  any  buffer  h  in  the  range  1  <  h  <  H(x )  is  denoted 
by  W  : 


(17) 


IK'  =  max  lK(x,  h) 

i, A:  \<h<H{t) 


The  maximum  window  size  for  any  session  x  at  any  buffer  h  in  the  range 
2  <  h  <  H(x)+1  is  denoted  by  W”  : 


To  keep  each  buffer  h  in  the  range  1  <  h  <  H(x)+ 1  from  overflowing, 
the  following  restriction  is  placed  on  the  flow  over  hop  h  —  1  during  every  slot 
t  >  1  :  if  buffer  h  is  full  at  the  end  of  slot  t—  1  ,  the  window  is  said  to  be 
closed ,  and  session  x  may  not  transmit  a  packet  over  hop  h  —  1  during  slot  t. 
In  other  words, 

(19)  B(x,  h ,  t— 1)  «  W(x,  h)  implies  P(z,  h  —  1,  1,  t)  =  0 

This  restriction  is  implemented  using  logical  quantities  called  permits.  Buffer  h 
has  W(x,  h)  permits  permanently  associated  with  it.  Every  packet  waiting  in 
buffer  h  holds  a  permit  for  buffer  h ,  and  any  leftover  permits  for  buffer  h  are 
stored  at  hop  h  —  1  .  A  packet  being  transmitted  over  hop  A  —  1  must  carry 
with  it  a  permit  for  buffer  h.  (If  none  are  available  at  hop  h  —  1  ,  then  the 
packet  may  not  be  transmitted.)  Whenever  a  packet  is  transmitted  over  hop  h 
(i.e.,  removed  from  buffer  h)  during  a  time  slot  t,  the  packet  relinquishes  its 
permit  for  buffer  h,  and  hop  h  returns  this  permit  upstream  to  hop  h  —  1 
during  that  same  slot  t.  t  The  return  of  permits  is  accomplished  in  the 

t  In  this  model,  link  transmission  is  error-free.  Hence,  it  is  possible  for  hop  h 
to  know,  at  the  beginning  of  a  slot  t,  that  a  packet  p  will  be  successfully 
transmitted  over  hop  h  during  slot  t.  Therefore,  hop  h  can  safely  send 
packet  p’s  permit  for  buffer  h  back  to  hop  h  —  1  during  slot  t.  In  a  real 
network,  with  imperfect  transmission,  the  error  control  process  (i.e.,  link 
layer  protocol)  for  hop  h  could  have  its  own  storage  area  where  the  packet  p 
is  transferred  during  the  slot  t  in  which  its  transmission  is  first  attempted. 
With  this  arrangement,  hop  h  could  still  send  packet  p’s  permit  back  to  hop 
h  —  1  during  slot  t  without  risking  buffer  overflow. 


following  way.  A  permit  for  buffer  1  is  returned  by  having  the  session’s  origin 
node  notify  the  session’s  source  (i.e.,  hop  0).  A  permit  for  a  buffer  h  in  the 
range  2  <  h  <  H[x)+ 1  is  returned  to  hop  h  —  1  by  transmitting  a  notice 
over  some  link  with  direction  opposite  to  hop  h—  1  .  This  notice  requires  few 
bits  and  can  be  piggybacked  onto  a  data  packet  if  any  are  available.  Therefore 
the  link  capacity  required  to  implement  permits  will  be  ignored,  f 

2.4  Link  Scheduling 

At  any  link,  packets  for  any  particular  session  are  transmitted  in  the  order 
in  which  they  arrived  from  their  preceding  hop.  Packets  from  different 
sessions,  however,  are  not  necessarily  transmitted  in  order  of  arrival.  Each  link 
has  a  scheduler  to  decide  which  session  will  use  the  link  during  each  time  slot. 
Various  scheduling  disciplines  are  possible.  This  section  describes  round  robin 
scheduling,  first-come-first-served  scheduling,  and  a  generalization  of  these 
disciplines  called  bounded  delay  scheduling. 

t  Consider  a  session  x  that  is  the  sole  user  of  each  link  in  its  path.  Suppose  x 
has  heavy  demand;  i.e.,  it  inserts  a  packet  into  buffer  1  whenever  that 
buffer  is  not  full.  If  x's  window  sizes  are  at  least  two,  then  its  long-term 
average  throughput  RA( x)  equals  one,  the  link  capacity.  However,  if 
lV(x,  h)  =  1  for  some  A,  then  RA{x)  can  be  no  more  than  Vz  .  For  this 
reason,  a  window  size  of  two  seems  to  be  the  smallest  practical  value. 


2.4.1  Round  Robin  Scheduling 

To  implement  a  round  robin  discipline,  the  scheduler  for  link  /  consults  a 
fixed  data  structure  consisting  of  session  identifiers  arranged  in  a  directed  ring. 
Each  session  using  l  appears  exactly  once  on  this  round,  robin  ring.  The  link 
scheduler  also  maintains  a  variable  called  the  ring  position  that  points  to  some 
session  on  the  ring.  Whenever  a  session  x  transmits  a  packet  over  l  in  a  time 
slot,  the  ring  position  is  updated  to  x  during  that  slot. 

Consider  the  scheduling  of  slot  t  at  link  l.  Let  session  x  be  the  ring 
position  at  the  end' of  slot  t—  1  .  Let  y  be  the  session  immediately  following  x 
on  the  ring.  During  slot  t,  the  link  scheduler  searches  the  ring,  starting  with 
y,  until  it  finds  the  first  session  z  that  has  both  packet(s)  and  permit(s) 
available;  i.e.,  z  must  satisfy 

B(z,  h,  t—  1)  >  0 

and 

B(z,h  +  l,t-l)  <  W(z,h  + 1) 

where  h  is  the  hop  number  of  link  /  for  z.  A  packet  for  session  z  is 
transmitted  over  /  and  the  ring  position  is  updated  to  z  during  slot  t.  If  the 
ring  is  searched  through  session  x  without  success,  then  the  search  stops  after 
x,  the  ring  position  remains  at  x,  and  nothing  is  transmitted  over  l  during  slot 
t. 

Each  session  examined  in  the  search  described  above  is  said  to  have  been 
offered  one  chance  to  use  link  l.  Let  C'(x ,  l.  s ,  t)  denote  the  number  of 
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chances  offered  to  session  x  at  link  l  during  interval  (s,  t]  .  If  A  is  the  hop 
number  of  link  l  for  session  x,  then  C(x,  A,  s,  t )  is  defined  to  equal 
C'(x,  l,  s,  t )  .  For  s  >  t  ,  define  C’(x,  l,  s,  t)  and  C(x,  h,  s,  t )  to  be  zero. 


2.4.2  First-Come-First-Served  Scheduling 

First-come-first-served  link  scheduling  is  complicated  by  the  window  flow 
control  mechanism.  With  this  discipline,  a  packet  waiting  in  buffer  A  of 
session  x,  1  <  A  <  H(x )  ,  seizes  a  permit  for  its  next  buffer  A  + 1  as  early  as 
possible.  Note  that  the  packet  may  enter  buffer  h  before  or  after  the  permit 
needed  for  buffer  A  +  l  arrives.  Once  both  the  packet  and  its  permit  for 
buffer  A  +  l  have  arrived  at  bop  A,  the  packet  is  said  to  be  authorized,  for 
transmission  over  hop  A,  and  a  future  transmission  slot  on  that  link  is  reserved 
for  that  session. 

First-come-first-served  scheduling  transmits  packets  over  a  link  in  order  of 
their  authorization  times.  To  this  end,  the  scheduler  for  each  link  l  maintains 
a  first-in-first-out  transmitter  queue.  If  a  packet  for  session  x  becomes 
authorized  to  use  l  during  slot  t,  then  a  reservation  for  x  is  added  to  the  tail  of 
the  transmitter  queue  during  slot  t.  .Associated  with  link  l  is  a  fixed  tie¬ 
breaking  list  ;  each  session  using  /  appears  exactly  once  in  this  list.  If  packets 
for  several  sessions  become  authorized  to  use  l  during  slot  t ,  then  their 
reservations  are  added  to  the  tail  of  the  transmitter  queue  during  slot  t  in  the 
order  in  which  the  sessions  appear  in  l's  tie-breaking  list.  At  the  beginning  of 


slot  f  +  1  ,  the  link  scheduler  for  l  notes  which  session  holds  the  reservation  at 
the  head  of  the  transmitter  queue.  That  reservation  is  removed  from  the  queue 
and  a  packet  for  that  session  is  transmitted  over  /  during  slot  J  +  l  . 

Consider  a  session  x  that  uses  link  l  as  its  hop  h,  1  <  h  <  H[x)  .  Since  x 
can  have  at  most  W(x,  h)  packets  and  W(x,  h+ 1)  permits  for  buffer  h  + 1 
waiting  at  hop  h,  the  number  of  reservations  for  x  in  l's  transmitter  queue 
(i.e.,  the  number  of  authorized  packets)  can  be  no  more  than 
min[W(x,  h),  W^XjA  +  l)]  ;  note  that  f 

(20)  min  (  W(x,  h),  W{x,  A  +  l)  ]  <  W(x,  h  +  1)  <  W"  <  oo 

2.4.3  Bounded  Delay  Scheduling 

This  section  describes  a  family  of  link  scheduling  disciplines  called  bounded 
delay  disciplines.  Consider  a  link  l  used  by  at  least  one  session.  The 
scheduling  discipline  for  l  is  said  to  have  bounded  delay  if  there  exists  a 
positive  integer  A'(l)  (called  a  schedule  delay  bound)  such  that,  for  all  sessions 
x  using  l  and  for  all  packets  p  >  1  of  x, 

x  Window  size  U'(x.  //(x)+l)  was  required  to  be  finite  so  that  the  number  of 
packets  for  session  x  in  the  transmitter  queue  at  hop  H(x)  would  be 
bounded  even  in  the  case  where  /f(x)  =  1  and 
H'(x,  H(x ))  =  U'f-r*  1)  =  cc  .  It  will  be  shown  in  Section  2. -1.3  that 
bounded  transmitter  queues  make  first-come-first-served  a  bounded  delay 
discipline;  such  disciplines  offer  delay  and  throughput  guarantees  to  be 
studied  in  Chapters  3  and  5. 


-  31  - 


IVW5T5* m .’I ■  W*9IP 'J* im  3 -A  3 


(21) 

T(x,  A,  p) 

<  max  [  T(x,  A  — 1,  p),  T(x,  A,  p  —  1),  T(x,  A  +  l,  p—  W(x,  A  +  l))  ]  +  A'{1 ) 

where  A  is  the  hop  number  of  link  l  for  session  x,  1  <  A  <  H(x)  .  In  other 
words,  once  packet  p  is  in  buffer  A,  and  there  are  no  packets  older  than  p  in 
buffer  A,  and  there  is  room  in  buffer  A  +  l  ,  packet  p  is  guaranteed  to  be 
transmitted  over  hop  A  within  A'(l)  time  slots.  Apart  from  satisfying  (21), 
the  scheduling  decision  for  a  slot  t  at  link  l  may  be  any  deterministic  function 
of  time  t  itself,  of  the  initial  levels  B(x,  A,  0)  of  all  buffers  A  of  all  sessions  x, 
0  <  A  <  H(x)+ 1  ,  and  of  the  throughputs  P(x,  A,  r  — 1,  r)  of  all  sessions  x 
over  all  hops  A,  0  <  A  <  H(x)+ 1  ,  at  all  past  times  r ,  1  <  r  <  t  . 

Note  that  round  robin  is  a  bounded  delay  discipline,  with  A'(l)  =  N'(l)  . 
To  see  that  first-come-first-served  also  has  bounded  delay,  consider  a  session  x 
that  uses  link  l  as  its  hop  A.  Consider  a  packet  p  >  1  of  x  waiting  in  buffer 
A.  Suppose  there  are  no  packets  for  x  in  buffer  A  that  are  older  than  p,  and 
suppose  the  window  for  buffer  A  +  l  of  session  x  is  open.  Consequently,  p  is 
authorized  for  transmission  over  hop  A.  Consider  how  many  packets  could  be 
ahead  of  p  in  link  f s  transmitter  queue.  It  was  given  that  there  are  no  packets 
for  session  x  ahead  of  p.  There  can  be  at  most  Ar,(/)— 1  sessions  other  than  i 
using  /.  It  was  explained  in  Section  2.4.2  that  a  session  can  have  no  more  than 
IV"  packets  in  any  link  transmitter  queue.  Therefore,  the  number  of  packets 
ahead  of  p  in  /’ s  transmitter  queue  is  no  more  than  (A:,(/)—  !]•  H'"  .  In 


>>\-i S: 


addition  to  the  packets  ahead  of  it,  packet  p  itself  must  be  transmitted  within 
the  A'{1)  time  slots.  Hence,  first-come-first-served  is  a  bounded  delay 
discipline,  with  A'(l )  =  [N'(l)—l]‘W"  +  1  .  An  example  of  a  link  scheduling 
discipline  without  bounded  delay  is  a  priority  scheme  where  the  session  with 
highest  priority  could  monopolize  a  link  for  an  indefinite  period  of  time. 

In  a  system  where  each  link  l  has  a  bounded  delay  scheduler  with  delay 
bound  A'(l)  ,  define  A(x,  h)  to  equal  A'(l )  if  link  l  is  hop  h  for  session  x. 
Also  define  A{x)  to  be  the  largest  schedule  delay  bound  at  any  link  in  the 
path  of  session  x  : 

(22)  A(x)  =  max  A(x,  h) 


2.5  Demand 

Recall  that  packet  arrivals  for  a  session  x  are  modeled  as  occasional  services 
at  hop  0.  This  section  explains  the  session  demand  model  in  more  detail. 
During  random  time  slots  t  >  1  ,  the  session  source  (i.e.,  hop  0)  attempts  to 
place  one  packet  (taken  from  the  infinite  supply  in  buffer  0)  into  buffer  1.  If 
the  window  for  buffer  1  is  closed,  i.e.,  if  B(x,  1,  t—  1)  =  lt'(x,  1)  ,  then  the 
attempt  fails  and  the  packet  transfer  does  not  take  place.  The  number 
of  such  attempts  (called  chances  at  hop  0)  during  interval  (s.  t]  is 


denoted  by  C(x,  0,  s,  t)  .  For  s  >  t  ,  C(x,  0,  s,  t)  is  defined  to  be 


-  33- 


zero.  f  The  long-term  average  number  of  chances  per  time  slot  for  cession  x  at 
hop  0  (if  this  average  exists)  is  called  the  demand  rate  X(x)  .  The  results  of 
Chapters  3  and  5  do  not  require  the  existence  of  this  average  for  every  session. 
Note  that  X(x)  <  1  .  % 

The  sample  space  of  the  demand  model  is  denoted  by  Q.  A  single  sample 
point  u  in  n  determines  an  entire  sample  path  of  demand  for  the  whole 
network;  i.e.,  u>  determines  C(x,  0,  £ —  1,  t )  for  all  sessions  x  and  all  times 
t  >  1  .  Since  the  demand  is  the  only  random  element  in  the  system  model,  a 
single  sample  point  u  in  D  also  determines  the  evolution  of  the  entire  system 
after  time  0.  The  cr-algebra  for  the  demand  model  is  the  one  generated  by 
events  of  the  following  type:  C(x,  0,  t—1,  t)  is  specified  for  a  single  session  x 
and  a  single  time  t  >  1  ,  while  the  demand  for  other  sessions  and  other  times 
is  arbitrary. 

Different  sections  in  Chapters  3,  4  and  5  make  different  assumptions  about 
the  probability  measure  of  the  demand  model.  Some  theorems  require  the 
demand  of  each  session  to  be  extremely  regular,  almost  deterministic.  Other 


*  Although  C(x,  h,  s,  t )  is  defined  for  h  >  0  only  in  the  context  of  round 
robin  link  scheduling,  C[x,  0,  s,  t)  is  defined  here  regardless  of  the  link 
scheduling  discipline. 

{  A  session  x  whose  actual  demand  rate  is  greater  than  one  packet  per  slot  can 
be  modeled  with  X(x)  =  1  ,  since  the  network  cannot  offer  x  a  throughput 
rate  greater  than  one  packet  per  slot  even  if  x  is  the  sole  user  of  each  link  in 
its  path. 
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results  permit  the  demand  process  C(x,  0,  t—  1,  t)  of  a  session  x  to  be 
Bernoulli.  One  section  assumes  only  that  the  times  between  chances  for  a 
session  at  hop  0  are  independent  and  identically  distributed.  Some  results 
require  that  the  demand  processes  of  the  various  sessions  be  independent; 
others  permit  dependence.  One  theorem  makes  no  demand  assumptions  at  all. 
The  one  set  of  demand  assumptions  under  which  all  the  results  of  Chapters  3, 
4  and  5  hold  is  the  heavy  demand  assumption,  viz.,  that  C[z,  0,  t  —  1,  t)  =  1 
for  all  sessions  x  and  all  times  t  >  1  . 


2.6  System  Specification 


A  system  is  fully  specified  by  describing  the  following  items:  the  network 
topology,  the  set  of  sessions  using  the  network,  the  sessions’  paths,  the  initial 
buffer  levels,  the  window  sizes,  the  scheduling  discipline  (e.g.,  round  robin  or 
first-come-first-served),  the  schedule  parameters  (e.g.,  the  rings  for  round  robin 
scheduling  or  the  tie-breaking  lists  for  first-come-first-served  scheduling),  the 
initial  schedule  state  (e.g.,  the  initial  ring  positions  for  round  robin  scheduling 
or  the  initial  transmitter  queues  for  first-come-first-served  scheduling),  and  the 


probability  measure  of  the  demand  model. 


2.7  Miscellaneous  Bounds 


Some  bounds  on  quantities  defined  in  this  chapter 
reference. 

1  <  H{x)  <  H  <  oo 

1  <  N'(l)  =  N{x,  h)  <  Ar(x)  <  N  <  S  <  oc 

1  <  A'(l)  —  A(x,  h )  <  A(x)  <  oo 

W(x ,  0)  =  oc 

1  <  W(x,  1)  <  oo 

1  <  W(x,  h)  <  oo 

2  <  V^(x,i7(i)+1)  <  oo 
W(x,  h)  <  W' 

1  <  W'  <  oo 
lf(x,  /i)  <  VK" 

2  <  IK"  <  cc 

0  <  B{x,  h,  t)  <  U'(x.  h) 

0  <  B(x,  H(x)+ 1,  t)  <  1  <  W(x,  H(x)+ 1) 


are  listed  below  for  easy 


for  all  x 

for  all  /,  x,  h  such 
that  link  /  is  hop  h 
for  session  x 

for  all  /,  x,  h  such 
that  link  l  is  hop  h 
for  session  x 

for  all  x 

for  all  x 

for  all.x, 

for  2  <  h  <  H(x) 
for  all  x 
for  all  x, 

for  1  <  h  <  H{x) 


for  all  x, 

for  2  <  h  <  H(x)+  1 


for  all  x, 

for  0  <  h  <  HU). 
for  t  >  0 

for  all  x,  for  <  >  0 
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B(x,  0 ,  t)  =  oo 

for  all  x,  for  t  >  0 

B(x,  h,  t)  <  oo 

for  all  x, 

for  1  <  A  <  A/(x)+ 1, 
for  i  >  0 

VI 

o 

P(x,  A,  s,  t)  —  P'(x,  l,  s,  t)  <  i  —  s 

for  all  x,  A,  /  such 
that  link  /  is  hop  A 
for  session  x, 
for  i  >  s  >  0 

0  <  P(x,  0,  s,  t)  <  t  —  s 

for  all  x, 
for  £  >  5  >  0 

0  <  P(x,  ff(x)+l,  s,  f)  <  t  —  5 

for  all  x, 
for  t  >  s  >  0 

VI 

o 

C(x,  A,  s,  i)  =  C"(x,  l,  s,  t)  <  t  —  $ 

for  all  x,  A.  1  such 
that  link  l  is  hop  A 
for  session  x, 
for  t  >  s  >  0 

0  <  C(x,  0,  5,  t)  <  t  —  s 

for  all  x, 
for  f  >  s  >  0 

0  <  <  1 

for  all  x  whose  avg. 
throughput  exists 

0  <  X(x)  <  1 

for  all  x  whose  avg. 
demand  exists 

0  <  T(x,  h,  p)  <  oo 

for  all  x, 

for  0  <  A  <  #(x)+ 1. 
for  all  integers  p 

1  <  =  (x,  p)  <  cc 


for  all  x, 

for  all  p  >  1  such 
that  T(x,  1.  p )  <  oc 


3.  PACKET  DELAY 


This  chapter  studies  the  cross-network  delay  of  packets  for  a  particular 
session  x  in  a  system  with  bounded-delay  link  scheduling.  The  window  sizes 
W(x,  h)  for  buffers  h  in  the  range  2  <  h  <  H(x)+ 1  are  assumed  to  be  at 
least  two  but  finite.  The  capacity  W(x,  1)  of  buffer  1  is  arbitrary,  possibly 
even  infinite.  The  window  sizes  of  the  other  sessions  in  the  network  are 
arbitrary.  The  demands  of  the  sessions,  including  x,  are  arbitrary;  session 
demand  rates  need  not  exist.  Theorem  1  shows  that  the  cross-network  delay 


for  each  packet  of  x  is  at  most 


'ff(z) 

E 

A  — 2 


W(x,  h) 


■A(x)  +  l.f 


It  was  explained  in  Section  2.4.3  that  round  robin  scheduling  and  first- 
come-first-served  scheduling  are  bounded  delay  disciplines,  with  schedule  delay 
bounds  A'(l)  of  N'(l)  and  N'(l)'  W”—  W"+\  ,  respectively.  Therefore,  the 
cross-network  delay  bounds  of  Theorem  1  for  round  robin  systems 


and  first-come-first-served  systems  are 


H(z) 

E 

A  —  2 


W(x,  h) 


•N(x)  +  1 


and 


t  If  the  average  throughput  and  the  average  cross-network  delay  per  packet 
exist  for  session  x,  then  Little’s  formula  [19,  4]  gives  the  following  upper 
bound  on  the  average  cross-network  delay  per  packet: 


Hlz) 

1  +  £  h ) 


A-2 


1 


This  bound  may  be  tighter  than  the  bound  of 


Theorem  1.  Note,  however,  that  Theorem  l’s  bound  applies  to  each  packet 
of  session  x. 


W(x,  h)^[N(x)'W"— W"+l]  +  1,  respectively.  Example  1  shows  a 

round  robin  system  where  one  packet  of  session  x  has  a  cross-network  delay 
that  matches  the  bound  of  Theorem  1.  Example  2  shows  a  first-come-first- 

served  system  where  one  packet  of  session  x  has  a  cross-network  delay  of 
’h(x)  1 

2  W(:r,  h)  —  1  '[N^yW'—W+l]  +  1  ,  which  is  close  to  the  bound  of 

A- 2 

Theorem  1.  f  Clearly,  the  delay  guarantees  afforded  by  this  theorem  for  round 
robin  systems  are  superior  to  those  for  first-come-first-served  systems.  (It  is 
not  being  claimed  that  round  robin  scheduling  always  offers  lower  packet 
delays  or  fairer  packet  delays  than  first-come-first-served  scheduling.) 


t  In  Example  2,  it  is  critical  that  W(x,  H(x))  =  2  . 


3.1  Theorem  1:  Bound  on  Packet  Delay 


Suppose  a  system  has  been  specified  that  satisfies  the  assumptions  of 
Chapter  2  and  uses  a  bounded  delay  scheduling  discipline.  Let  x  be  some 
session.  The  capacity  W(x,  1)  of  buffer  1  of  x  is  arbitrary  —  it  may  even  be 
infinite.  Suppose  that 

(23)  2  <  W'(x,  h)  <  oo  for  2  <  h  <  H(x)+ 1 

The  window  sizes  of  the  other  sessions  in  the  network  are  arbitrary  (i.e.,  these 
window  sizes  only  need  to  satisfy  the  basic  assumptions  of  Chapter  2).  The 
demands  of  the  sessions,  including  session  x,  are  arbitrary.  It  follows  that,  for 
each  packet  p  >  1  of  session  x  such  that  T(x,  0,  p)  <  oo  , 

(24)  T(x,  h,  p)  <  oo  for  1  <  h  <  f/(x)+l 


(25) 


E(x,  p)  < 


H(x) 

E 

h- 2 


•A(x)  +  1 


In  other  words,  every  packet  of  x  that  enters  buffer  1  eventually  leaves  the 
network  and  has  bounded  cross-network  delay. 


Proof  of  Theorem  1 

Let  us  clarify  the  scheduling  assumptions.  It  follows  from  (21)  that,  for  all 
packets  p  >  1  of  x  and  all  hops  h  in  the  range  1  <  h  <  H(x)  , 


T{x,  h,  p ) 

<  max  [  T(x,  h  —  1,  p),  T(x,  h,  p  —  1),  T(x,  A  +  l,  p—  W(x,  h  +  l))  ]  +  A(x,  h) 

(26) 

<  max  [  T(x,  h  —  1,  p),  T(x,  h,  p  —  1),  T(x,  h  +  l,  p—  W(x,  h  +  l))  ]  +  A(x) 
Also  recall  this  assumption  about  the  operation  of  hop  H(x)+ 1  : 

(27)  T(x,  H{x)+ 1,  p)  =  T(x,  H(x),  p)  +  1  for  p  >  1 

Now  it  will  be  proved  by  contradiction  that  (24)  holds  for  each  packet 
p  >  1  of  session  x  such  that  T(x,  0,  p)  <  oo  .  If  this  were  not  the  case,  then 
there  would  be  some  smallest  positive  integer  p  such  that 

(28)  T(x,  0,  p )  <oo 
and 

(29)  T(x,  h,  p)  =  oo  for  some  h,  1  <  h  <  #(x)+l 
Since  p  is  the  smallest  such  value, 

(30)  T(x,  h,  p)  <oo  forO<A<  H(x)+ 1,  all  integers  p  <  p 

Let  h  be  the  smallest  integer  in  the  range  1  <  h  <  i7(x)+ 1  for  which  (29) 
holds;  i.e., 

(31)  T(x,  h.  p)  =  co 
and 

(32)  T(x,  h,  p)  <  co  for  1  <  h  <  h 


Now  it  will  be  shown  that 


ss 


(33)  T(x,  A,  p)  <  oo 

There  are  three  cases  to  consider.  If  h  =  1  ,  then  (33)  follows  from  (26),  (28) 
and  (30).  If  2  <  h  <  #(x)  ,  then  (33)  follows  from  (26),  (32)  and  (30).  If 
h  =  H(x)+1  ,  then  (33)  follows  from  (27)  and  (32).  Hence  (33)  is  proved.  Note 
that  (33)  contradicts  (31).  This  completes  the  proof  of  (24). 

Next  (25)  will  be  proved.  Let  p'  >  1  be  any  packet  of  session  x  such  that 
T(x,  0,  p')  <  oo  ;  p'  will  be  fixed  for  the  remainder  of  this  section.  By  (24), 
T(x,  1,  p')  <  oo  ,  so  the  cross-network  delay  of  packet  p'  is  well-defined.  It 


must  be  shown  that 


H(x) 

=(i,?')  <  2  W(x,h)  *A(x)  +  1 

A- 2 


If  H(x )  —  1  ,  then  (34)  follows  immediately  from  definition  (13);  so  assume 


H{x)  >  2 


First,  let  us  justify  the  following  claim  for  integers  h  and  p. 


(36)  T(x,  h,  p)  <  T(x,  1,  p’)  for  1  <  h  <  H{x),  p  <  p’-  V  W{x,  h) 


Inequality  (36)  follows  from  the  buffer  capacity  constraints: 
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?{x,  1,  p')  >  T(x,  2,  p'-W{x,  2)) 

>  T(x,  3,  p'- ^(x,  2)- W(x,  3)) 


>  T(x,  A,  p'-  £  VT(x,  A)) 

A-  2 

>  T(x,  A,  p) 


Note  that  (36)  holds  if  p  <  1  (in  which  case  T(x,  A,  p)  **  0).  Moreover,  (36) 
holds  even  if  packet  p’  is  farther  downstream  than  buffer  1  at  time  0  (in  which 
case  every  element  in  the  chain  of  inequalities  above  equals  zero). 


Next,  let  us  define  ©(A,  p)  for  each  hop  A  of  x  in  the 
1  <  A  <  H{ x)  and  every  integer  p  : 


(37)  ©(A,  p)  -  T(x,  1,  pr)  +  max 


0,  p-p'+tf(x)-A+  ^  W{x,h) 


A-2 


*.4(x) 


Note  that 


(38)  ©(A,  p  —  1) 
and 

(39)  ©(A,  p  — 1) 


©(A  +  l,  p-  W{x,  A  +  l))  for  1  <  A  <  H{x)~  1.  all  p 


©(A.  p)  —  A(x)  if 


p-p'+H{x)~  A  +  V  W(x.  A  ' 


A  -  2 


>  1 


Also  note  that,  by  (37)  and  (23), 


(40)  ©(A,  p-1)  >  ©(A  —  1,  p)  for  2  <  A  <  //( j),  all  p 


Let  us  prove  the  following  claim: 


T(x,  A,  p)  <  ©(A,  p)  for  1  <  A  <  H(x),  p'+ 1—2*  VF(x,  A)  <  p  <  p' 

4- 2 


The  proof  is  by  induction  on  p.  The  base  cases,  viz., 


H(x)  H{  i) 

p'+l-2*  2  W(x,  A)  <  p  <  p'-£W(x,A) 

A— 2  A-2 


are  easy  to  prove:  by  (36)  and  definition  (37), 


T(x,  A,  p)  <  T(x,  l,p')  <  ©(A,  p) 

for  these  values  of  p  and  for  1  <  A  <  H(x )  .  For  the  induction  step,  consider 
an  integer  p  in  the  range 


p'+l-  £  W(x,  A)  <  p  <  p> 
4-2 


The  induction  hypothesis  asserts  that 


T(x,  A,  p)  <  ©(A,  p)  for  l<A<//(x),  p'+l  — 2-  £  VF(x,  A)  <  p  <  p  — 1 

A-2 


It  must  be  shown  that 


T(x.  A.  p)  <  ©(A,  p)  for  1  <  A  <  //(. 


First  (44)  will  be  proved  for  small  values  of  A.  Let  A'  be  the  largest  hop  in 

A' 

the  range  1  <  A'  <  H(x)~  1  such  that  ^  (x,  A)  <  p'—p  .  By  (36)  and 

A-2 
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definition  (37), 

(45)  T(z,  A,  p)  <  T(x,  1,  p')  <  ©(A,  p)  for  1  <  A  <  h' 

For  hops  A  in  the  range  A'  <  A  <  H(x)  ,  the  proof  of  (44)  will  be  by 
induction  on  A.  The  base  case  A  =  A'  is  covered  by  (45)  above.  For  the 
induction  step,  consider  a  hop  A  (if  any)  in  the  range 

(46)  A'+l  <  A  <  H{x)~  1 

(The  case  A  *  H(x )  will  be  treated  separately.)  The  induction  hypothesis 
asserts  that 

(47)  T(x,  A  — 1,  p)  <  ©(A  — 1,  p) 

It  must  be  shown  that 

(48)  T(x,  A,p)  <  ©(A,  p) 

If  p  <  1  ,  the  proof  of  (48)  is  trivial,  since  T(x,  A,  p)  =  0  in  this  case.  If 
p  >  1  ,  first  apply  (26)  and  induction  hypotheses  (47)  (for  the  induction  on  A) 
and  (43)  (for  the  induction  on  p ): 

r(x,  i,  p) 

W  .  .  .  . 

<  max  [  T(x,  A  —  1,  p ),  T(x,  A ,  p  —  1).  T(x.  A  +  1.  p  —  ll'(x,  h  +  1))  J  +  .4(x) 

^  . 

<  max  (  ©(A  —  1,  p),  ©(A,  p  —  1),  ©(A  +  l,  p—  W(x,  A  +  l))  ]  +  A(x) 

Now  apply  (40),  (38)  and  (39)  to  (50)  to  reach  the  desired  conclusion  (48): 
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T(x,  h,  p)  <  ©(A,  p  —  1)  +  ^.(s) 

-  ©(A,p) 

The  proof  for  the  remaining  case,  viz.,  h  =  H(x )  ,  is  similar,  but  (23)  and  (12) 
are  used  to  show  that  the  term  T(x,  A  +  l,  p—  W(x,  A  +  l))  in  (49)  is  not 

greater  than  the  term  T(x,  h,  p  —  1)  .  The  proof  of  this  case  will  not  be 

presented.  This  completes  the  proof  of  (44)  by  induction  on  h ,  thereby 
completing  the  proof  of  (41)  by  induction  on  p. 

The  desired  conclusion  (34)  follows  from  definition  (13),  (41),  and  definition 
(37): 

S(x,  p')  -  T(x,  H(x),  p')  -  T(x,  1,  p')  +  1 

<  e{H{x),  p>)  -  T(x,  1,  p')  +  1 

=  ^  W(x,  h)  ’A(x)  +  1 

i-2 


This  completes  the  proof  of  Theorem  1. 


-  46- 


3.2  Example  1:  Round  Robin  Scheduling 

Consider  a  system  that  satisfies  the  assumptions  of  Chapter  2  and  has  the 
layout  shown  in  Figure  1.  The  system  includes  a  session  x  for  which  H(x)  >  2 
and  N(x )  >  2  .  Session  x  uses  links  ,  /2  ,  ...  ,  ^h{x)  >  that  order.  (For 
each  of  these  links,  there  is  another  link  with  opposite  direction  that  is  not 
shown  in  Figure  1  and  is  used  only  to  return  flow  control  permits.)  Sessions 
Vi  ,  j/o  >  —  >  Vn(x)—  i  use  only  link  x)  .  Round  robin  link  scheduling  is  used. 
The  ring  position  for  Ih{x)  time  0  is  x.  The  window  size  for  buffer  1  of 
session  x  is  at  least  two  and  may  be  either  finite  or  infinite.  The  window  sizes 
for  buffers  2  through  H(x)  of  session  x  are  at  least  two  but  finite,  and  these 
buffers  are  initially  full.  For  each  session  ,  y2  ,  ...  ,  Vn{x)~i  >  the  capacity  of 
buffer  1  is  at  least  two,  and  this  buffer  is  initially  nonempty,  f  Every  session  in 
the  system  has  heavy  demand;  i.e., 


tin  practice,  this  "initial"  system  state  could  arise  if  sessions  r,  j/[  ,  i 

3/o  >  •••  ,  J/jV(i)-i  started  before  time  0,  when  there  were  already  many  other  \ 

sessions  using  link  ,  and  if  these  extra  sessions  terminated  at  time  0.  j 

i 


! 


(51) 


C(x,  0,  t- 1,  t )  =  C(yt  ,  0,  t- 1,  0 
=  £7(^2  »  0,  £ — 1,  t ) 


^{y n(z)—  i  >  £  i»  0 

=  i 

for  all  times  t  >  1  . 

The  evolution  of  this  system  is  simple.  Session  x  transmits  packets  over 
link  Ih[z)  during  slots  N(x),  2 N(x),  3 N(x),  ....  Let  p'  be  the  youngest 
packet  of  session  x  in  buffer  2  at  time  0;  i.e., 

H(z) 

(52)  p'  -  £  A) 

h-2 

Packet  p'  is  transmitted  over  during  slot  p'mN(x)  .  Therefore,  by 

definition  (13), 

5(x,  p')  =  T(x,  H(x),  p')  -  T(x,  1,  p’)  +  1 
=  p'*Ar(x)  —0+1 

*.V(x)  +  1 

This  matches  the  upper  bound  (25)  of  Theorem  1. 

Things  are  slightly  different  for  packets  p  >  pf  of  session  x.  Because  of 


'h(z) 

(53)  =  ^  »■’(*.  A) 

A- 2 
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the  extreme  initial  conditions,  the  system  experiences  a  mild  transient.  No 
permits  for  buffer  2  of  session  x  are  returned  to  hop  1  until  time  slot 
N(x)+H(x)—  2  ,  after  which  one  permit  is  returned  every  N(x)  slots. 
Therefore,  for  each  packet  p  >  p'  of  session  x, 

T(x,  1,  p)  =  [N(x)  +  H{x)  —  2]  +  {p  —  p'  —  l)*iV(x)  +  1 

(54)  =  (p  -  p')'N(x )  +  H{x)  -  1 

Link  lmx)  ,  however,  functions  periodically  even  from  time  0,  and 

(55)  T(x,  H(x),  p)  =  p-N(x) 

By  definition  (13),  (54),  (55),  and  definition  (52), 

=(x,  p )  =  T(x,  H{x),  p)  -  T(x,  1,  p)  +1 

=  pr'N(x)  —  H(x )  +  2 
'  [*(*) 

(56)  =  ^  W{x,h) 

LA-2 

Comparing  (56)  with  (53)  shows  that  the  cross-network  delay  for  each  packet 
p  >  p'  is  slightly  less  than  the  delay  for  packet  p'  . 

For  future  reference,  note  that  the  long-term  average  throughput  of 


•iV(x)  +1  —  [H(x)  —  1] 


session  x  is  l/.Y(x)  . 


3.3  Example  2:  First-Come-First-Served  Scheduling 

Consider  a  system  that  satisfies  the  assumptions  of  Chapter  2  and  has  the 
layout  shown  in  Figure  2.  The  system  includes  a  session  x  for  which  H(x)  >  3 
and  N(x)  >  2  .  Session  x  uses  links  /x  ,  /2  ,  •••  ,  l h(z )  »  in  that  order.  (For 
each  of  these  links,  there  is  another  link  with  opposite  direction  that  is  not 
shown  in  Figure  2  and  is  used  only  to  return  flow  control  permits.)  Sessions 
Vi  ,  y«  ,  ...  ,  Un(x)—  i  use  only  link  Ih(x)-i  •  Sessions  z1  ,  z2  ,  ...  ,  use 

only  ^f(r)  •  The  window  size  for  buffer  1  of  session  x  is  at  least  two  and  may 
be  either  finite  or  infinite.  The  window  sizes  for  buffers  2  through  H(x)—l  of 
session  x  are  at  least  two  but  finite,  and  these  buffers  are  full  at  time  0.  The 
initial  level  of  buffer  H(x )  of  session  x  is  one,  and  W(x,  H{x ))  =  2  .  Buffers 
1  and  2  of  sessions  yx  ,  y2  ,  ...  ,  yN{z)~  l  »  *i  *  z2  »  —  »  2JV(z)-i  ^ave  capacity 
W"  >  2  ;  buffer  1  for  each  of  these  sessions  is  initially  full,  and  buffer  2  is 
initially  empty.  First-come-first-served  link  scheduling  is  used.  Session  x 
appears  last  in  the  tie-breaking  lists  of  links  ancl  ^H(z)  •  The 

transmitter  queues  at  links  lx  ,  /2  ,  ...  ,  l h(z)-2  are  empty  at  time  0,  because  of 
a  lack  of  permits  for  session  x.  Initially,  the  transmitter  queue  for  i 

contains  IF"  reservations  for  each  session  yx  .  y2  •  •••  ,  Cin  an.v  oi'der) 

followed  by  one  reservation  for  session  x.  (Although  x  has  F  {x ,  H(x)—l) 
packets  waiting  to  be  transmitted  over  hop  H{x)—  1  ,  only  one  of  these  has  a 
permit  for  buffer  H(x)  .  Hence  x  has  only  one  reservation  in  the  transmitter 
queue  for  hf(x)- 1  •)  Initially,  the  transmitter  queue  for  lf^x) 


contains  IF" 


reservations  for  each  session  zx  ,  z2  ,  ...  ,  zn(z)~  i  0D  any  order)  followed  by 
one  reservation  for  session  x.  Note  that  the  initial  transmitter  queue  lengths  at 


i  and  Ih(x)  are  N[x)%W"—Wn+ 1  .  f  Every  session  in  the  system  has 
heavy  demand;  i.e., 

(57)  C(x,  0,  £  —  1,  t)  =  C{yx  ,  0,  t-l,  t) 


“  N(z)—  1  t  ^  If  0 

-  C(zlt  0,  f-l,  0 


=*  i  ?  0)  t  i)  0 

=  l 


for  all  times  t  >  1  . 

The  evolution  of  this  system  is  simple.  Session  x  transmits  packets  over 
links  1  and  during  slots  [fV(x)*  \V"—  VV'"+ 1], 

2-[.V(x)-lK"-  W"+l],  3*[.V(x)*  \V"—  W"+l] .  Let  p'  be  the  youngest 


t  In  practice,  this  "initial”  system  state  could  arise  if  sessions  x. 
yx  ,  ...  ,  t/,v(x)-i  »  z\  >  •••  i  zN(z)—  i  started  at  various  times  before  time  0. 
when  there  were  already  many  other  sessions  using  links  Ih[z)-2  •  ^H(z)- 1 
and  lff(z )  ,  and  if  these  extra  sessions  terminated  at  time  0. 
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packet  of  session  x  in  buffer  2  at  time  0;  i.e., 

H(x) 

(58)  p'  «  £  ^(*.  M  “  1 

A  —  2 

Packet  p'  is  transmitted  over  during  slot  p'-[jV(x)*  W7'—  VK"+ 1]  . 

Therefore,  by  definition  (13), 

=(x,  p')  =  T(x,  JJ(s),  p')  -  T(x,  1,  p')  +  1 


-  p''[N{xyw"-w"+ 1]  -  o  +  i 


(59) 


’ff(x) 

E 

A-2 


W(x,  A)  -  1 


*[;V(x)‘  W-  IV" + 1] 


+  1 


This  is  almost  equal  to  the  upper  bound  (25)  of  Theorem  1. 


Things  are  slightly  different  for  packets  p  >  p'  of  session  x.  Because  of 
the  extreme  initial  conditions,  the  system  experiences  a  mild  transient.  No 
permits  for  buffer  2  of  session  x  are  returned  to  hop  1  until  time  slot 
[N(x)"  W1'— W"+l]+ H(x)—3  ,  after  which  one  permit  is  returned  every 
A'(x)*  W"—  W"+ 1  slots.  Therefore,  for  each  packet  p  >  p'  of  session  x, 

T(X,  1,  p)  =  [[.V(x)-U'"  -  l V"  +  1]  +  H{x)  -  3] 

+  (p  -  p'  -  iy[.x{iyiv"  -  w"  +  1]  +  1 


(60)  -  (p  -  p'y[N{x)-W"  -  W"  +  1]  +  H{ x)  -  2 

Links  hf(z)- 1  and  h/(z)  >  however,  function  periodically  even  from  time  0. 


j*  ^  .*  ,*  s 

A  A  A  /.A 


and 
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(61) 


T (x,H(x),p)  =  p'[N(x)‘  W"  —  W"  +  1] 


By  definition  (13),  (60),  (61),  and  definition  (58), 
2(x,  p)  =  T (x,H(x),p)  -  T(x,  1 ,  p)  +1 


-  p'-[N(xy\V"  -  W"  +  lj  -  H(x)  +  3 


(62) 


H(x) 

E  A)  -  1 


[N(x)-W"  -  W"  +  1]  +  1 


-  \H{x)  -  2] 


U*-*  i  ) 

Comparing  (62)  with  (59)  shows  that  the  cross-network  delay  for  each  packet 
p  >  p'  is  slightly  less  than  the  delay  for  packet  p'  . 

For  future  reference,  note  that  the  long-term  average  throughput  RA{x)  of 
session  x  is  l/[N(x )•  W"—  W"-l-l]  .  f 


fin  this  example,  it  is  critical  that  \V(x,  H(x))  =  2  .  The  long-term  average 
throughput  and  the  cross-network  delay  for  session  x  could  be  significantly 
improved  by  increasing  W(x,  H(x))  . 


V  *.■  V.V.V.V.'V.V.V. 'Vf lV.V -A  •  V 


■  *  >  _■  / 


[>.' 
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4.  SESSION  THROUGHPUTS  IN  SYSTEMS  WITH  LARGE  WINDOWS 

This  chapter  studies  the  fairness  of  session  throughputs  in  systems  where 
the  window  size  W(x,  h )  is  finite  but  very  large,  for  each  buffer  h  of  each 
session  x  in  the  range  1  <  h  <  H(x)  .  The  exact  assumptions  about  the 
window  sizes  vary  from  section  to  section.  Round  robin  link  scheduling  is 
assumed  throughout  the  chapter,  except  in  Section  4.6,  where  it  is  shown  that 
certain  round  robin  results  do  not  hold  if  first-come-first-served  scheduling  is 
used  instead.  This  chapter  assumes  that  each  session  x  has  a  well-defined,  real 
demand  rate  X(x)  in  the  range  0  <  X(x)  <  1  ,  f  but  the  detailed  demand 
assumptions  vary  among  the  sections. 

The  chapter  is  organized  as  follows.  The  max-min  flow  criterion,  which  is 
taken  as  the  definition  of  throughput  fairness  throughout  this  chapter,  is 
described  in  Section  4.1.  According  to  this  criterion,  each  session  has  a  unique 
fair  throughput  rate.  Section  4.2  contains  some  preliminary  results  needed  in 
later  sections.  Theorem  2  of  Section  4.3  analyzes  a  system  during  an  interval 
[Tx  ,  To)  of  smooth  demand.  Specifically,  Theorem  2  assumes  that  there 
exists  a  constant  A  such  that  the  demand  of  each  session  x  over  each 
subinterval  (s,  t\  of  (T {  ,  T 2)  is  within  A  packets  of  the  nominal  amount 

t  An  example  of  a  session  x  with  demand  rate  X(x)  =0  is  a  session  that  has 
only  a  finite  number  of  chances  at  hop  0  after  time  0  and  therefore  injects 
only  a  finite  number  of  packets  into  the  network. 
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X(x)*(£  — s)  .  It  is  also  assumed  that  most  window  sizes  are  at  least 
Z'(H+  1)S‘NS~  1*(A+2)  .  Theorem  2  concludes  that  the  throughput  of  each 
session  x  over  each  subinterval  of  (Tx  ,  T2)  is  within 
(H+  l)5*./V2'5- 1#(  M/,+3A+4)  packets  of  the  fair  amount,  regardless  of  the 
length  of  the  subinterval. 

A  steady-state  analysis  is  found  in  Section  4.4.  This  section  assumes  that 
there  exists  a  constant  A  such  that  the  demand  of  each  session  i  over  each 
interval  (s,  f]  is  within  A  packets  of  the  nominal  amount  \(x)*(£  —  s)  . 
Again,  most  window  sizes  are  assumed  to  be  at  least  Z'(H+l)s  'Ns~l'(A+ 2)  . 
Corollary  1  of  Theorem  2  concludes  that  the  long-term  average  throughput 
Ra(x)  eacb  session  x  equals  its  fair  rate.  Theorem  3,  the  steady-state 

analog  of  Theorem  2,  shows  that  there  exists  a  time  TSS  >  0  such  that  the 
throughput  of  each  session  x  over  each  interval  later  than  Tss  is  within 
(H+  1)S'NS~  1,(A+2)  packets  of  the  fair  amount,  regardless  of  the  length  of 
the  interval,  f  Note  that  this  bound  on  throughput  unfairness  in  steady  state 
is  tighter  than  the  transient  bound  of  Theorem  2  and  is  independent  of  \V  . 
Section  4.4  also  contains  several  corollaries  of  Theorem  3  dealing  with  steady- 
state  buffer  levels. 

t  Although  no  upper  bound  is  known  for  the  length  Tss  of  the  transient 
period,  Theorem  2  shows  that  the  throughput  of  each  session  j  during  the 
transient  period  is  within  {H+  1)S’N2S~  1*(  W7+3A+ 4)  packets  of  the  fair 
amount. 


Theorem  4  of  Section  4.5  uses  a  burstier  demand  model:  the  sessions  are 


assumed  to  have  independent  Bernoulli  demand  processes.  Most  window  sizes 
are  assumed  to  be  at  least  12'(H+1)S 'Ns~*  and  at  least  a  certain  fraction  a 
of  W' .  Theorem  4  concludes  that  (with  probability  one)  the  long-term 
average  throughput  Rj\(x)  each  session  x  differs  from  its  fair  rate  by  no 


more  than 


745-(ff+l)25-iV25-1 


.  In  other  words,  the  session  throughput 


a*(  W'f's 

rates  can  be  made  arbitrarily  close  to  the  fair  rates  by  choosing  window  sizes 
that  are  of  the  same  order  of  magnitude  and  are  sufficiently  large. 


Theorems  2,  3  and  4  show  that  enormous  windows  (of  comparable  size)  are 
sufficient  to  guarantee  fair  (or  nearly  fair)  throughput  rates.  One  wonders 
whether  large  windows  are  actually  necessary.  Section  4.4  presents  an  example 
where  the  throughput  rates  are  quite  unfair  unless  very  large  windows  are  used. 
Things  could  be  worse,  however:  Section  4.6  shows  that  if  first-come-first- 
served  link  scheduling  is  used  instead  of  round  robin  scheduling,  then  even 
large  windows  cannot  guarantee  throughput  fairness.  Chapter  5  determines 
what  throughput  guarantees  are  possible  in  systems  with  small  windows. 


4.1  Fairness  Criterion 


This  section  desc-:bes  the  max-min  flow  criterion,  which  will  be  taken  as 
the  definition  of  throughput  fairness  throughout  Chapter  4.  The  version  of  the 
criterion  presented  here  was  proposed  by  Hayden  [13].  A  similar  criterion  was 
developed  independently  by  Jaffe  [16,  17].  Later,  Gafni  and  Bertsekas  [6] 
phrased  the  principle  more  economically  and  generalized  it.  The  criterion  is 
described  here  as  it  applies  to  the  system  model  presented  in  Chapter  2.  In 
particular,  it  is  assumed  that  the  sessions  and  routes  have  been  specified  and 
that  all  links  have  unit  capacity.  It  is  also  assumed  that  each  session  x  has  a 
well-defined,  real  demand  rate  X(x)  in  the  range  0  <  X(x)  <  1  .  f 

First,  let  us  define  some  terms.  An  allocation  r  is  a  function  that  assigns 
each  session  x  a  real  rate  r(x)  in  the  range  0  <  r(x)  <  X(x)  without 
violating  the  link  capacities.  In  other  words,  the  sum  of  the  rates  for  all 
sessions  sharing  a  link  l  cannot  exceed  the  link’s  capacity: 

(63)  £  r(x)  <  1 

i  using  l 

The  full  rate  list  of  an  allocation  r  is  a  unique  vector  consisting  of  the  rates 
r(x)  assigned  to  all  the  sessions  x.  If  the  same  rate  is  assigned  to  k  different 

|  It  is  easy  to  generalize  the  max-min  flow  criterion  to  systems  where  the  link 
capacities  and  the  session  demand  rates  are  arbitrary  nonnegative  real 
numbers. 


sessions,  then  that  rate  appears  k  times  in  the  full  rate  list.  The  components 
of  the  full  rate  list  must  appear  in  nondecreasing  order.  The  reduced  rate  list 
of  an  allocation  is  formed  by  deleting  duplicate  values  from  the  full  rate  list. 

Now  fairness  can  be  defined.  An  allocation  r  satisfies  the  max-min  flow 
criterion  if  no  other  allocation  has  a  full  rate  list  that  is  lexicographically 
greater  than  the  full  rate  list  of  r.  Roughly  speaking,  this  means  that  the 
smallest  rate  assigned  to  any  session  by  r  is  as  large  as  possible  and,  subject  to 
that  constraint,  the  second-smallest  assigned  rate  is  as  large  as  possible,  etc. 
Each  of  these  nested  optimization  problems  can  be  formulated  as  a  linear 
program  [13],  and  it  is  not  difficult  to  show  that  there  exists  a  unique  allocation 
that  solves  them  all.  The  rates  assigned  in  this  unique  max-min  allocation  will 
be  called  the  fair  rates.  The  objective  of  Chapter  4  is  to  determine  conditions 
under  which  the  long-term  average  throughput  of  each  session  x  equals 

its  fair  rate. 

Let  I  denote  the  length  of  the  reduced  rate  list  for  the  max-min  allocation 
(i.e.,  the  number  of  distinct  fair  rates),  and  let  Rp(i)  denote  the  ith  element 
of  this  list.  Any  session  whose  max-min  fair  rate  is  Rp{i)  is  said  to  have 
congestion  index  i.  For  example,  all  sessions  with  the  smallest  fair  rate  have 
congestion  index  1,  and  all  sessions  with  the  largest  fair  rate  have  congestion 
index  I.  Let  /(x)  denote  the  congestion  index  of  session  x.  In  other  words, 
the  fair  rate  for  session  x  is  Rp(I(x))  . 


The  max-min  flow  criterion  can  also  be  stated  in  terms  of  bottlenecks. 
Suppose  some  allocation  is  given  (not  necessarily  the  max-min  allocation).  A 
link  l  is  called  a  bottleneck  link  for  a  session  x  using  l  if  the  rate  r(x)  assigned 
to  x  is  at  least  as  large  as  the  assigned  rate  of  any  other  session  using  l,  and  if 
the  entire  capacity  of  l  is  assigned  to  the  sessions  using  it.  The  following 
equivalence  is  not  difficult  to  prove:  an  allocation  satisfies  the  max-min  flow 
criterion  if  and  only  if,  for  each  session  x,  either  r(x)  =  X(x)  (i.e.,  the  demand 
of  x  is  a  bottleneck)  or  x  has  at  least  one  bottleneck  link. 

Once  the  max-min  criterion  has  been  stated  in  terms  of  bottlenecks,  it  is 
easy  to  see  why  round  robin  link  scheduling  might  be  expected  to  achieve  the 
max-min  fair  rates  [7,  9].  Consider  a  session  whose  demand  exceeds  its 
throughput.  Packets  for  this  session  should  accumulate  at  the  input  to  its 
most  congested  link.  Therefore,  the  session  should  never  have  to  forfeit  its 
turn  in  that  link’s  round  robin.  This  ensures  that  the  session’s  average 
throughput  will  be  at  least  as  large  as  that  of  any  of  its  competitors  at  that 
link,  and  it  also  ensures  that  the  link  will  stay  busy.  Thus  the  link  should  be  a 
bottleneck  link  for  that  session  in  the  technical  sense  defined  above.  Every 
session  that  is  not  limited  by  its  own  demand  should  have  such  a  bottleneck 
link;  hence  the  resulting  average  throughputs  should  equal  the  max-min  fair 
rates.  Of  course,  this  crude  plausibility  argument  does  not  constitute  a  proof. 

For  the  remainder  of  this  thesis,  the  term  "bottleneck"  will  be  used  to  mean 
"bottleneck  with  respect  to  the  max-min  allocation."  In  other  words,  a  link  /  is 


a  bottleneck  link  for  a  session  x  using  l  if 


(64) 


l{y)  <  Hz)  for  ail  sessions  y  using  l 

E  Rf(I( y))  -  1 

y  using  l 


If  a  bottleneck  link  for  session  x  has  hop  number  h,  1  <  h  <  H(x)  ,  then  h  is 
a  bottleneck  hop  for  x.  Hop  0  is  said  to  be  a  bottleneck  hop  for  x  if 


(65)  **(/(*))  =  M*) 

i.e.,  if  session  x  is  bottlenecked  by  its  demand.  Every  session  x  has  at  least  one 
bottleneck  hop  h  in  the  range  0  <  h  <  H(x )  .  (Hop  H(x)+ 1  is  never  said  to 
be  a  bottleneck  hop.) 

For  future  reference,  let  us  define  Rq{ x,  h )  and  R'c(x,  l )  for  a  session  x 
that  uses  a  link  l  as  its  hop  h,  1  <  h  <  H(x)  :  f 

i  -  E  *FV(y)) 

(66)  Rc(x,h)  =  0  -  - S\Z(x‘!l)  | - 

where  Y{x,  l)  is  the  set  of  sessions  y  using  l  for  which  I(y)  <  I(x)  ,  and 
Z(x,  l )  is  the  set  of  sessions  z  using  l  for  which  I(z)  >  I(x)  .  Note  that 

t  It  will  be  shown  in  Sections  4.3.1  and  4.4.3  that  if  round  robin  scheduling  i: 
used,  and  if  the  session  demands  are  sufficiently  regular,  and  if  the  window 
are  large  enough,  then  Rf^(x,  l)  is  a  lower  bound  on  the  rate  at  which  t h < 
round  robin  scheduler  for  link  l  offers  chances  to  session  x. 


(67)  Rc(x,  h)  > 


> 


Rf(I(x)) 


E  RfW*)) 

z6Z{x,  l) _ 


Z(x,l)  | 


E  */■(/(*)) 

zS  Z(x,  l) _ 

i  W,  o  i 


If  A  is  a  bottleneck  hop  for  x,  then  equality  holds  throughout  (67).  If  h  is  not 
a  bottleneck  hop  for  x,  then  one  or  both  of  the  inequalities  of  (67)  must  be 
strict.  Let  us  also  define  Rq(x,  0)  for  each  session  x: 


(68)  Rc(x ,  0)  =  X(x) 
Note  that 

(69)  Rc(x,  0)  >  Rf(I(x)) 


and  that  hop  0  is  a  bottleneck  for  session  x  if  and  only  if  equality  holds  in  (69). 
In  summary,  for  any  hop  h  of  any  session  x  in  the  range  0  <  h  <  H{x)  , 

Rc(xi  h)  —  Rf(I(x))  if  h  is  a  bottleneck  hop  for  x 

Rc(x,  h)  >  Rf(I(x))  if  h  is  not  a  bottleneck  hop  for  x 


The  concepts  of  this  section  will  now  be  illustrated,  using  the  system  of 
Figure  3  as  an  example.  The  network  contains  links  ,  l2  ,  /3  ,  and  l4  .  (For 
each  of  these  links,  there  is  another  link  with  opposite  direction  that  is  not 
shown  in  Figure  3  and  is  used  only  to  return  flow  control  permits.)  Each  link 
has  unit  capacity.  Session  x^  uses  only  link  .  Session  x._>  uses  lx  followed  by 
U  .  Session  x3  uses  all  four  links.  Sessions  x4  and  x5  use  only  /3  .  Sessions  x6 
and  x7  use  only  l4  .  Every  session  has  a  demand  rate  of  1,  except  session  x4  . 
whose  demand  rate  is  1/6  ,  and  session  x6  ,  whose  demand  rate  is  1/3  .  The 
max-min  fair  rate  for  each  session  is  1/3  ,  except  session  ,  whose  fair  rate  is 
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1/6  ,  and  session  x5  ,  whose  fair  rate  is  1/2  .  The  full  rate  list  for  the  max-min 
allocation  is  (  1/6  ,  1/3  ,  1/3  ,  1/3  ,  1/3  ,  1/3  ,  1/2  )  .  The  reduced  rate  list  for 
the  max-min  allocation  is  (1/6, 1/3, 1/2).  Session  x4  has  congestion  index 

1.  Session  x5  has  congestion  index  3.  The  other  sessions  have  congestion  index 

2.  Sessions  xx  and  x2  have  link  / j  as  a  bottleneck.  Session  x3  has  two 
bottleneck  links  —  lx  and  /4  .  The  bottleneck  hop  for  x4  is  hop  0  —  its  demand. 
Link  /3  is  the  bottleneck  for  x5  .  Session  x6  has  its  demand  and  f4  as 
bottlenecks.  Link  /4  is  also  the  bottleneck  for  x7  .  Link  U  is  not  a  bottleneck 
for  any  session,  since  it  has  unused  capacity. 

Note  that,  in  this  example,  the  max-min  allocation  does  not  maximize  the 
sum  of  the  session  rates  or  minimize  idle  link  capacity.  The  max-min  flow 
criterion  is  certainly  not  the  only  reasonable  capacity  allocation  strategy  one 
could  propose.  For  some  applications,  one  might  be  willing  to  tolerate  some 
unfairness  in  order  to  achieve  more  efficiency. 


4.2  Preliminary  Results 


This  section  contains  various  lemmas  needed  for  Sections  4.3  and  4.4. 
Lemma  2  (Section  4.2.2)  assumes  an  upper  bound  on  the  throughput  of  sessions 
whose  congestion  index  is  less  than  a  particular  value  t;  this  bound  is  used  to 
derive  a  lower  bound  on  the  number  of  chances  offered  by  a  link’s  round  robin 
scheduler  to  a  session  with  congestion  index  i.  Lemma  6  assumes  lower  bounds 
on  the  demand  of  a  particular  session  and  on  the  number  of  chances  the 
session  receives  at  each  link  in  its  path;  these  bounds  and  the  assumption  of 
large  windows  are  used  to  derive  a  lower  bound  on  the  throughput  of  the 
session.  The  proof  of  Lemma  6  requires  Lemmas  3,  4  and  5;  all  four  of  these 
are  found  in  Section  4.2.3.  Lemma  7  (Section  4.2.4)  assumes  an  upper  bound 
on  the  demand  of  a  particular  session  x  and  a  lower  bound  on  the  throughput 
of  sessions  whose  congestion  indices  are  I(x)  or  less;  these  bounds  and  the 
properties  of  max-min  fairness  (viz.,  the  existence  of  bottleneck  hops)  are  used 
to  derive  an  upper  bound  on  the  throughput  of  session  x.  Lemma  1  (Section 
4.2.1)  notes  various  inequalities  relating  the  functions  0  »  epl{±'  0 , 

Epu(A,  i)  ,  FCL(£,i),  FPL(A,i),  FPU(£,i),  F"PU(£,i),  DCL(£.i.k). 
DPi( A.  i ,  I\)  .  and  DPi-{ A.  /.  K )  defined  below.  The  argument  A  is  a  real 
number,  and  i  and  K  are  integers.  (The  subscripts  C,  P,  L.  and  U  stand  for 
"chances,"  "packets,"  "lower  bound,"  and  "upper  bound,"  respectively.  The 
subscripts  indicate  how  the  functions  will  be  used.) 
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(71)  Ecl( A,  0 

(72)  Epl( A,  0 

(73)  EPU( A,  t) 

(74)  FCL{ A,  0 

(75)  Fpi(A,  i ) 

(76)  FPU(A,  i) 

(77)  F'WA,  0 


( H  +  -  1  )»iV2,-3*( W'  +  2A  +  2) 

0 

for  i  >  2 

for  *  <  1 

(if  +  1)’-(7V  -  l)*./V2,'-3-(W''  +  2A  +  2) 

0 

for  t  >  2 

for  i  <  1 

(if  +  1),'-JV2,‘-1*(M'1  +  2A  +  2) 

0 

for  i  >  1 

for  »  <  0 

(if  +  ly-^JV’-^A  +  2)  -  1 

0 

o  o 

-J 

•*,  «*, 

IA  IV 

O  h- 

(if  +  l)‘*iV,~1*(A  +  2)  -  1 

0 

for  t  >  1 

for  t  <  0 

(if  +  1)*‘-Af,'“1-(A  +  2) 

0 

for  *  >  1 

for  t  <  0 

(if  +  1)‘-Af‘*(  V^'  2A  +  2) 

0 

for  t  >  1 

for  i  <  0 

(78)  Dcl(A,  t,  K)  —  £’c£,(A,  t)  +  K'Fcd&,  *) 

(79)  Dpi(A,  i,  K)  =  Epd A.  0  +  K’Fpl(A,  *) 

(80)  Dpu{ A,  *,  AT)  =  Epy(&,  t)  +  K'FPU(A,  i) 


4.2.1  Lemma  1:  Miscellaneous  Inequalities 


The  following  inequalities  hold  for  all  positive  integers  i  and  K  and  all 


nonnegative  real  numbers  A  . 

(81) 

Fpl(A,  i) 

> 

0 

(82) 

*>  K—l) 

> 

0 

(83) 

Fpu(&>  *—1) 

> 

0 

(81) 

Dpu(A,  * — 1,  K) 

> 

0 

(85) 

Fpl(A,  i+1) 

> 

F PL  (^*  0 

(86) 

Dpl{&i  *  +  l>  K) 

> 

Dpl(^i  *»  K) 

(87) 

Fpui^i  *’+l) 

> 

Fpui&i  *) 

(88) 

Dpu[A,  »  +  l,  K) 

> 

Dpui&i  *>  K) 

(89) 

FCl(*>  0 

> 

A 

(90) 

DCL{X  i,  K) 

> 

K'A 

(91) 

Fcl{&>  0 

> 

{N-  1)’FPU(A,  i-1)  +  1 

(92) 

F cl  (^»  *»  K) 

> 

(N-  1)-DPU(A,  t'—l,  AT)  +  A- 

(93) 

Fpl{&’  0 

> 

(ff  +  1)*Fcl(A,  i)  +  if 

(91) 

Fpi  ( A ,  t,  K) 

> 

(tf  +  «,  A-)  +  K-H 

(95) 

FPU{A.  i) 

> 

F pl{ -X  0  +  1 

(96) 

F”pu{A,  i ) 

> 

(.V  -  1)-/>l(A.  0  +  W'H  +  A 

(97) 

Dpui-^'  *i  K) 

> 

(A  —  l)'DPi(A,  i,  1)  +  Dpl{ A,  A'— 1) 

+  W'H  +  A 

These  inequalities  follow  directly  from  definitions  (71)  -  (80);  the  proofs  will 
not  be  presented. 


4.2.2  Lemma  2:  Lower  Bound  on  Chances,  given  Upper  Bound  on 


Throughput 

Suppose  a  system  has  been  specified  that  satisfies  the  assumptions  of 
Chapter  2  and  uses  round  robin  scheduling.  Suppose  each  session  has  a  well- 
defined  demand  rate.  Let  x  be  some  session,  and  let  l  be  some  link  used  by  x. 
Let  K  be  a  positive  integer,  and  let  sx  ,  ,  s2  ,  t2  ,  ...  ,  ,  tK  be  times 

satisfying  0  <  <  s2  <  f2  ^  *  *  *  <  SK  ^  *K  •  Let 

(98)  G  >  0 

be  a  real  number  such  that,  for  each  session  y  using  l  with  I(y)  <  I{x )  , 

(99)  £  P\y,  l,  sk  ,  tk)  <  Rp(I(y)y  £  (tk  —  sk)  +  G 

i-l  k-l 

It  follows  that 

(100)  £  C'(x,  l,  sk  ,  tk)  >  R'c[x,  1)’  £  (tk  -  sk)  —  (N  —  1)*G  -  K 

k-l  k-l 


Proof  of  Lemma  2 

Let  Y  denote  the  (possibly  empty)  set  of  sessions  y  on  l  for  which 
I(y)  <  I{x)  .  Let  Z  denote  the  set  of  sessions  z  on  /  for  which  I(z)  >  I(x)  . 
Note  that  Z  includes  x.  For  k  =  1,  2,  ...  ,  K  ,  let  qk  be  the  number  of  slots  in 


( sk  ,  tk]  that  are  not  used  by  sessions  in  Y: 


(101) 
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Qk  =  (h  -  Sk)  ~  E  P'(V’  z>  s*  t  y 

*er 

Since  these  qk  slots  are  not  used  by  Y,  the  round  robin  scheduler  at  /  will  offer 
each  of  these  slots  to  at  least  one  session  in  Z;  hence 

Qk  <  E  C'(2>  sk  ,  **) 

26  z 

(102)  <  |  Z  |  •  max  C'(z,  l,  sk  ,  tk) 

z£  2 


By  the  operating  rules  of  the  round  robin  scheduler,  session  x  must  receive 
almost  as  many  chances  as  any  other  session  on  l  during  ( sk  ,4]  ;  in 

particular, 


(103)  C'{x,  l,  sk  ,  tk)  >  max  C'(z,  l,  sk  ,  tk)  -  1 

z€  Z 


Combining  (103),  (102)  and  (101)  yields: 


(104)  C\x,  l,  sk  ,  tk)  > 


{tk  -  sk)  -  E  p,(y>  sk  .  tk) 

yer 


-  1 


Summing  over  k  yields: 


(105) 

E  c'(i,  1,  st .  4)  > 

k -  1 


E(4  -  **) 

km  1 


K 


E  E  p,{'J-  L  sk '  tk) 

y€Y  1 


-  7v 


Since  /(y)  <  /(i)  for  all  y  in  1',  assumption  (99)  can  be  substituted  above  to 

yield: 


E  c'(x>  l>  sk  >  h) 

k-  1 


> 


Z| 


a: 


E(4  -  **)  -  E 

it—  1  y&Y 


^(/(y))'E(4  -  **)  +  c 
*- 1 


(106) 


Z  | 


1  -  (/(*)) 
yer 


•  £  dt  ~  *t)  -  4|f ' -c  -  * 

*-l  I Z  I 


Applying  definition  (66)  and  assumption  (98)  gives  the  desired  result  (100): 


£  c(i,  i,  «» ,  tt)  >  «'c(*.  o-  £  («» -  «*)  -  -Lff •<?  -  ^ 

t-i  jfc-i  z  I 


-  HL.n  - 


K 


>  R'ci*>iy  E(fc-'*)  -  m-<? 

i-1 


«■ 


>  *'<?(*♦  0*  E  («*  -  st)  -  (*v  -  l)’G  -  K 

k-  1 


This  completes  the  proof  of  Lemma  2. 


4.2.3  Tandem  Queues  with  Finite  Buffers 

This  section  derives  a  lower  bound  on  the  throughput  of  a  session  from 
given  lower  bounds  on  the  session’s  demand  and  on  the  number  of  chances  the 
session  receives  at  each  link  in  its  path.  The  problem  is  difficult  because  the 
session’s  buffers,  while  large,  are  finite.  The  problem  is  solvable,  however, 
because  the  session’s  demand  and  chance  processes  are  fairly  smooth. 

Let  us  begin  with  slightly  oversimplified  sketches  of  the  results  of  this 
section  and  their  proofs.  The  key  result  is  Lemma  3.  This  lemma  focuses  on  a 
particular  buffer  h  of  the  given  session  x  in  the  range  1  <  h  <  H(x)  .  A 
lower  bound  is  assumed  for  the  throughput  over  hop  h  —  1  (i.e.,  into  buffer  h ) 
during  any  interval  when  buffer  h  is  not  full.  This  bound  has  a  special  form. 
It  is  the  product  of  a  nominal  rate  r  and  the  length  of  the  interval,  minus  a 
constant  error.  A  similar  lower  bound,  with  the  same  nominal  rate  r,  is 
assumed  for  the  throughput  over  hop  h  (i.e.,  out  of  buffer  h)  during  any 
interval  when  buffer  h  is  not  empty.  In  other  words,  it  is  known  how  the 
subpaths  upstream  and  downstream  of  buffer  h  behave  when  isolated  from 
each  other.  The  capacity  W{x,  h )  of  buffer  h  is  assumed  to  be  at  least 
slightly  larger  than  the  sum  of  the  error  constants  in  the  throughput  bounds 
for  the  two  subpaths.  Lemma  3  uses  the  given  throughput  bounds  for  the 
subpaths  in  isolation  to  derive  lower  bounds  of  the  same  form  that  apply 
during  any  interval,  regardless  of  the  level  of  buffer  h.  The  error  constants  in 
these  bounds  for  the  integrated  system  are  only  slightly  larger  than  the  sum  of 


the  error  constants  in  the  bounds  for  the  isolated  subpaths. 


The  proof  of  Lemma  3  is  structured  as  follows.  To  bound  the  throughput 
over  hop  h  —  1  during  an  interval  (s,  i]  ,  this  interval  is  divided  into  various 
subintervals.  There  is  an  initial  subinterval  in  which  buffer  h  moves  from  its 
initial  value  to  nearly  empty.  There  is  a  final  subinterval  in  which  buffer  h 
moves  from  nearly  empty  to  its  final  value.  Between  the  initial  and  final 
subintervals  are  subintervals  that  alternate  between  two  types.  During  a  type 
1  subinterval,  buffer  h  is  not  empty,  and  its  level  moves  from  nearly  empty  to 
nearly  full.  During  a  type  2  subinterval,  buffer  h  is  not  full,  and  its  level 
moves  from  nearly  full  to  nearly  empty.  During  a  type  2  subinterval,  the  given 
lower  bound  applies  to  the  throughput  of  the  isolated  upstream  subpath. 
During  a  type  1  subinterval,  the  given  lower  bound  applies  to  the  throughput 
of  the  isolated  downstream  subpath.  Moreover,  the  throughput  of  the 
upstream  subpath  during  a  type  1  subinterval  must  exceed  the  throughput  of 
the  downstream  subpath  by  approximately  W(x,  h)  packets,  since  buffer  h 
fills  during  a  type  1  subinterval.  The  window  size  W(x ,  h)  is  large  enough 
that  the  throughput  excess  (over  the  nominal  amount)  for  the  upstream 
subpath  during  a  type  1  subinterval  balances  the  possible  throughput  deficit 
(from  the  nominal  amount)  for  the  upstream  subpath  during  a  type  2 
subinterval.  Thus  the  throughput  over  hop  h  —  1  during  a  type  1/type  2  cycle 
is  at  least  r  times  the  combined  length  of  the  two  subintervals.  Net  deficits 


from  the  nominal  throughput  can  only  accrue  during  the  initial  and  final 


subintervals  of  the  interval  ( s ,  t ]  .  Hence  the  net  throughput  deficit  at  hop 
h  —  1  over  the  entire  interval  (5,  t]  cannot  be  too  great.  Lemma  3  analyzes 
the  throughput  over  hop  h  in  a  similar  manner. 

Lemma  4  is  a  simple  corollary  of  Lemma  3.  Lemma  4  assumes  a  lower 
bound  on  the  number  of  chances  received  by  a  particular  session  x  at  any  hop 
of  its  path  —  including  hop  0,  the  demand  hop.  This  bound  is  the  product  of  a 
nominal  rate  r  and  the  length  of  time  involved,  minus  a  constant  error. 
Sufficiently  large  windows  are  also  assumed.  For  each  buffer  h  of  x  in  the 
range  1  <  h  <  if(x)+l  ,  Lemma  4  proves  the  following  property:  during  an 
interval  when  buffer  h  is  not  full,  the  throughput  into  buffer  h  is  at  least  r 
times  the  length  of  the  interval,  minus  a  constant  error.  This  error  constant  is 
only  slightly  larger  than  the  sum  of  the  error  constants  in  the  given  chance 
bounds  for  hops  0  through  h—  1  .  The  proof  of  Lemma  4  is  by  forward 
induction  on  A,  using  Lemma  3  to  add  successive  hops  to  a  growing  upstream 
subpath. 

Under  the  same  assumptions  as  Lemma  4,  Lemma  5  derives  a  lower  bound 
on  the  throughput  out  of  each  buffer  h  of  x  during  intervals  when  the  buffer  is 
not  empty.  The  proof  of  Lemma  5  is  by  backward  induction  on  h.  using 
Lemma  3  to  add  successive  hops  to  a  growing  downstream  subpath. 

Lemma  6  makes  the  same  assumptions  as  Lemmas  4  and  5.  Since  Lemma  4 
analyzes  the  subpath  upstream  of  any  buffer  h  when  that  buffer  is  not  full, 
and  Lemma  5  analyzes  the  downstream  subpath  when  the  buffer  is  not  empty, 


Lemma  6  can  invoke  Lemmas  4,  5,  and  3  to  derive  a  lower  bound  on  the 


throughput  of  session  x  at  any  hop  during  any  interval,  regardless  of  the  buffer 
levels.  As  usual,  the  bound  is  the  procuct  of  r  and  the  length  of  the  interval, 
minus  a  constant  error.  This  error  constant  is  only  slightly  larger  than  the  sum 
of  the  error  constants  in  the  given  chance  bounds  for  hops  0  through  H(x )  . 
Of  Lemmas  3,  4,  5,  and  6,  only  Lemma  6  is  used  in  later  sections. 


4.2.3. 1  Lemma  3:  Lower  Bound  on  Throughput  of  Concatenated  Subpaths 


Suppose  a  system  has  been  specified  that  satisfies  the  assumptions  of 
Chapter  2  and  uses  any  scheduling  discipline.  Let  x  be  some  session.  Let  h  be 
some  hop  of  z  in  the  range  1  <  h  <  H[x )  .  Let  K  be  a  positive  integer,  and 
let  s1  ,  tl  ,  So  ,  to  ,  ...  ,  ,  tj{  be  times  satisfying 

0  S  j  t  ^  So  ^  to  ^  '  <  sK  <  lK  •  .Assume  that  the  following  two 

properties  hold  for  some  real  numbers  r,  e',  e”,  f,  and  f"  : 


(107)  If,  for  k  *  1,  2,  ...  ,  K  ,  J(k )  is  any  positive  integer,  and 

vk^  ,  tt/M-1  ,  ,  u^~~  ,  ...  ,  vk  ,  ut°  are  any  times  such  that 

sk  <  vJk[k)  <  <  «/(*)_2  <  •  •  •  <  V*1  <  <  tk 

J{k)  .  .  , 

and  such  that  i?(x,  h,  r)  <  W(x,  A)  for  all  r  in  (j  [t^  ,  u^-1)  ,  then 

1 


k  J{k)  .  K  J(k )  .  , 

E  EP(i,w,i<r')>  r-2  S(«r‘ -#/)-«’ 

jfc-iy-i  i-iy-i 


K 

E  J(k) 

k-  1 


(108)  If,  for  A  =  1,  2,  ...  ,  /C  ,  /(&)  is  any  positive  integer,  and 

1  ,  vk^~l  ,  ...  ,  uk  ,  u*1  are  any  times  such  that 

s*  <  <  v/W  <  1  <  VP*'  <  •  •  •  <  ukl  <  v£  <  tk 

J(k)  . 

and  such  that  B{x,  h,  r)  >  0  for  all  rin  (j  [«/  .  v£)  ,  then 

1 


K  J(k) 

E  Ep(x-h-ut 

k-l  j- I 


K 


r'E 

k-  1 


Jtk) 


j  -  1 


K 

E 


•/' 


Also  assume  that 


«  V  ’/  V  >  V 
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(109) 

/'+/"+  2  < 

lF(x,  h)  <  oo 

It  follows  that 

K 

K 

(110) 

E  p(x»  A_1»  sk  1  h)  > 

r •  E  (lk  ~  sk)  ~  e 

-  K'f 

k-  1 

i-  1 

and 

K 

K 

(111) 

E  P(x>  Sk  .  > 

1 

<0* 

1 

-V 

w 

• 

-  /C-/ 

Jt-1 

*-l 

where 

(112) 

C  = 

e'  +  cw 

(113) 

/  = 

/'+  /"+  1 

Proof  of  Lemma  S 

Inequality  (110)  will  be  proved  first.  For  each  k,  1  <  k  <  K  ,  let  us 
analyze  the  time  interval  [ sk  ,  tk]  separately.  The  first  step  is  to  break 
[s*  ,  tk]  into  various  subintervals.  Determine  a  positive  integer  J(k)  and 
define  times  uk  ,  vk  ,  uk  ,  vk  ,  uk  ,  ...  ,  vk^  ,  by  the  procedure  specified 

below.  Examples  are  shown  in  Figures  4  and  5. 


T 


«°  —  tk 

E:  j  «—  j  +  1 

■*—  earliest  time  v  in  [ sk  ,  w^-1]  that  satisfies 
B(x,  A,  t )  <  W{x,  A)  for  all  r  in  [u,  u^-1) 

ul  « —  earliest  time  u  in  [ sk  ,  v[\  that  satisfies 
B(x,  A,  t)  >  0  for  all  r  in  [u,  v{) 

if  uj.  >  sk  then  go  to  E 

J(k)  -  j 

It  is  not  difficult  to  verify  that  ul  and  t/j?  are  well-defined  and  that  this 
procedure  terminates.  Let  us  make  some  remarks  about  ul  and  vl  : 

(114)  sk  *  <  vk ^  <  uk^~l  <  v^~l  <  •  •  •  <  uk  <  vk  <  uk  = 

(115)  B(x,  h,  t)  <  W{x,  A)  for  all  r  in  [v/  ,  u/-1)  ,  1  <  j  <  /(A) 

(116)  A?(x,  A,  u/)  >  W(x,  A)  —  1  for  1  <  j  <  7(A)— 1 

(Note:  Strict  inequality  in  (116)  occurs  only  for  j  —  1 
and  only  if  f?(x,  A,  tk  —  l)  =  J9(x,  A,  £*)  =  Vf(x,  A)  .) 

(117)  If  dJk[k)  <  vl{k\  then  B{x,  h.vj[{k))  >  W{x,  h)  -  1 

(118)  B(x,  h ,  r)  >  0  for  all  r  in  [ul  .  r^)  .  1  <  j  <  /(A) 

(119)  B(x,  A,  u/)  =1  for  1  <  j  <  J(k)—l 


Now  the  facts  above  will  be  used  to  analyze  the  throughput  over  the 
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subintervals  (u/  ,  v/]  .  From  (116),  (119)  and  (10),  it  follows  that 

(120) 

P(x ,  A- 1,  4  ,  vi)  >  P(x,  A,  uj>  ,  v{ )  +  VF(x,  A)  —  2  for  1  <  j  <  J{k)-1 

To  develop  a  similar  inequality  for  j  =  /(£)  ,  first  let  us  justify  the  following 
claim: 

(121)  B( i,  A,  «/<*>)  <  B(i,  A,  «/<*>)  +  1 

If  tt/(U  “  ,  then  (121)  is  obviously  true.  If  ,  then  (121) 

follows  from  (117).  From  (121)  and  (10),  it  follows  that 

(122)  P{x,  A- 1,  u/W  ,  t;/^)  >  P(x,  A,  u/{t)  ,  v/W)  -  1 

Next,  the  throughput  over  the  entire  interval  ( sk  ,  tk]  can  be  studied.  By 

(114), 

P(x,  A  —  1,  sk  ,  tk )  —  P(x,  A  — 1,  «/<*)  ,  «°) 


(123) 

V(*) 

2  P{x,  A  — 1,  ui  ,  vi) 

+ 

’/(*)  , 

2  P(x,  A  —  1,  vi  ,  ui-1) 

y-i 

Applying  (120)  and  (122)  to  (123)  yields: 


(124) 


h—  l,  sk  ,  tk)  > 

Ji  k'l 

V  P(x.  /i.  ui  .  V"  ) 

+ 

J  <  .  .  , 

X]  P‘  I,  h  -  1.  r;  .  u;  ) 

y  ™  1 

+  [/(A)  -  1  ] ‘ [  O ’( x .  h)  -  2]  -  1 
Finally,  the  throughput  over  the  collection  ol  interval.'  1  -  ;  . 


.  ■ 


t  l 


,  ■OV..-V. 


_  .  A  -S  AA .  .  -V.  V. 


.'1 

Ai 


can  be  examined.  Summing  (124)  over  k  yields: 


(125) 


£  P(x,  h- 1,  Si 


k-  1 


.  h) 


> 


K  J(k) 

E  E  P{x,  h,  ul ,  4) 


k-  1  j'-l 


+ 


K  J(k)  . 

E  E  F(x>  A_1>  vi  *  ui  ) 

Jfc-1  ;'-l 


+ 


E  [-W  -  i] 

k-  1 


:[W(x,h)- 


2 


-  IC 


The  hypotheses  of  the  lemma  can  now  be  used  to  bound  the  right-hand  side  of 
(125).  It  follows  from  (114),  (118),  and  assumption  (108)  that 


K  J(k ) 

(126)  £  E  P{x,h,ul,vi)  >  r- 

’  K  i 

E  E  H  ~  «i) 

-  e"  - 

’  K 

£  A*) 

i  y-  i 

*-i y-i 

1 

Similarly,  it  follows  from  (114),  (115),  and  assumption  (107)  that 

(127) 

K  J(k) 

E  E  p(z’  /l~1>  vi  t  *)  >  r- 

k -  1  1 


K  J{k)  . 

K 

E  EW  -"!) 

-  e'  - 

E  J(k) 

k- l 

Substituting  (126)  and  (127)  into  (125)  yields: 
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K 

E  P(x,  h- 1,  sk  ,  tk)  >  r* 

k- i 


K  J{k ) 

E  EW  -“!) 

i-i y— i 


—  (e'  +  e")  -  K 


+ 


E  J(k) 

k-  1 

E  [-W  -  i] 

k-  1 


•(/'  +  /") 


W*f  A)-  2] 


(128) 


r-  £  (u*°  -  «/<*>)  -  ( e '  +  e")  -  /£•(/'  +  /"  +  1) 
Jfc-i 


+ 


£  («W  -  i] 


*- 1 


'[w(x,  h)  —  r-  r-  2] 


Applying  definitions  (112)  and  (113),  assumption  (109),  and  (114)  yields  the 
desired  result: 

E  **(x>  A_1>  sk  .  4)  >  E  (u*°  -  u/(*})  ~  e  ~  /c'/ 

k- 1  i-1 


=  r*  E  -<*)-«  ~  #'/ 

*-i 

This  completes  the  proof  of  inequality  (110). 

Now  (111)  will  be  proved.  For  convenience.  let  us  rephrase  assumptions 
(107)  and  (lOS)  by  juggling  the  superscripts  of  and  uk  : 
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(129)  If,  for  k  —  1,  2,  ...  ,  K  ,  J{k)  is  any  positive  integer,  and 

vim ,  %lw ,  „/(*)-. ,  „/<*>-> , ...  ,  vk  ,  uk  are  any  times  such  that 

Sk  <  vk <  uk^~l  <  *  *  '  <  vk  <  «*  £  lk 

J[k)  . 

and  such  that  B(x,  h,  r)  <  W(x,  h )  for  ail  r  in  (j  [v(  ,  u l)  ,  then 


^  4*)  K  A*),  V 

2  £F(i,M(^,«/)>  r-£  E 

*-  1  1  *-  1  1 


e'  — 


K 

E  A*) 


4-  1 


(130)  If,  for  &  =  1,  2,  ...  ,  if  ,  J{k)  is  any  positive  integer,  and 

uk ’(*)  ,  „/(*>-»  ,  «/(*)- 1  ,  ./(*)-*  ,  ...  ,  are  any  times  such  that 

s*  <  <  vk^~l  <  uk^~l  <  vk^~2  <  *  *  *  <  uk  <  vk  <  tk 

J(k) 

and  such  that  £(x,  A,  r)  >  0  for  all  rin  ,  v{  x)  ,  then 

y-i 


K  J(k )  .  ,  K  J(k) 

£  2/>( x,h,u{,v{-1)  >  r-£  E(vr'-ul)  -  e" 

4-  1  1  *-  l  1 


K 

E  -W 


4-  l 


The  proof  of  (111)  is  similar  to  that  of  (110).  As  before,  for 
k  =  1,  2,  ...  ,  /C  ,  the  interval  [sk  ,  f*]  must  be  broken  into  subintervals. 
Determine  a  positive  integer  J(k)  and  define  times  vk  ,  uk  ,  vk  ,  uk  , 

v( . ./>*>  ,  t;/^  by  the  procedure  specified  below.  Examples  are  shown  in 

Figures  6  and  7. 


ul  « —  earliest  time  u  in  [sk  ,  v{  *]  that  satisfies 
B(x,  h,  t)  >  0  for  all  rin  [u,  v/-1) 

vl  ■* —  earliest  time  v  in  [s*  ,  uj.\  that  satisfies 
B{ x,  h,  r)  <  W(x,  h)  for  all  rin  [v,  u{) 

if  vj.  >  sk  then  go  to  E 


J(k)  —  j 


The  following  properties  are  analogous  to  (114)  -  (119): 

(131)  sk  —  <  vk^_1  <  «/**>“ 1  <  •  •  •  <  vk  <  uk  <  vk  =  ^ 

(132)  B(x,  h,  r)  >  0  for  all  rin  [u/  ,  v^-1)  ,  1  <  <  /(&) 

(133)  B{x,  h,  u{)  <1  for  1  <  j  <  J{k)-1 

(Note:  Strict  inequality  in  (133)  occurs  only  for  j  *  1 
and  only  if  B(x,  h,  tk  —  1)  =  B(x,  h,  tk)  *  0  .) 

(134)  If  vk^k'1  <  uk^  ,  then  B(x,  h.  u/^)  <  1 

(135)  B{i .  h,  r)  <  U'(x,  h )  for  all  rin  [tq  .  ti')  .  1  <  j  < 

(13G)  B(x.  h.  i'i)  —  W\x,  h  )  —  1  for  1  <  j  <  J{k)—  1 


The  following  inequalities  can  be  proved  by  reasoning  similar  to  that  behind 


P{x,  h ,  v[  ,  ui)  >  P{x,  h  —  1,  v{  ,  u£)  +  W(x,  h)  —  2  for  1  < 


(138) 


P(x,  h,  „/<*>  ,  „/<*>)  >  P(x,h-l,viW,uiW)  - 


It  follows  from  (131),  (137),  and  (138)  that 


K 

E  p(*>  *  4)  = 

it- 1 


K  J{k) 

E  E^(*>  h,vl,ujl) 

k-  1  ;'-l 


+ 


'(*)  .  , 
E  E  ui  »  W  ) 

i-  1  J-  1 


(139) 


a:  j(t) 

E  E  p(x»  A_1»  W  »  ui) 

k-  1 


AT  /(*)  .  , 

E  Ef(x*  ui  >vl~  ) 

i  /- 1 


+ 


S  l-W  -  11 

*-  1 


[W(x,  h)  —  2} 


j  <  J{k)- 1 

1 


-  K 


Because  of  (131),  (135),  and  (132),  assumptions  (129)  and  (130)  can  be  applied 
to  (139)  to  yield: 


K 

E  P(x,  h,sk  ,tk)  >  r- 

k-  1 


K  J(k)  . 

E  SH'1-  H) 

k-  1  1 


+ 


K 

E  At) 

1 

e  \m  - 1] 

k-  1 


-  (e'+  e") 

*(/'  +  /") 

‘[W(x,  h)  -  2] 


-  K 


(140) 


K 


«  r*  £  K°  “  ~  (e'  +  e")  -  +  f"  +  l) 

i-l 


E  [•''(*)  -  il 

*-1 


W*.  h)~  f-  /"-  2] 


Applying  definitions  (112)  and  (113),  assumption  (109),  and  (131)  yields  the 
desired  result: 


E  P(*.  *.  .  <*)  >  r-£  (vt°  -  „/(*>)  -  e  -  /f-/ 

t-  l  k- 1 


“  r*  E  7*  -**)-«  -  #7 


i-I 


This  completes  the  proofs  of  inequality  (111)  and  Lemma  3. 
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4. 2. 3. 2  Lemma  4:  Lower  Bound  on  Throughput  of  Upstream  Subpath 

Suppose  a  system  has  been  specified  that  satisfies  the  assumptions  of 
Chapter  2  and  uses  round  robin  scheduling.  Let  x  be  some  session.  Let  Tx 
and  T2  be  times  satisfying  0<7,1<T2<oo.  (Note  that  T2  is  permitted 
to  be  infinite.)  Suppose  there  exist  real  numbers  r,  Gx  and  G2  such  that  the 
following  inequality  holds  for  every  hop  h  of  x  in  the  range  0  <  h  <  H(x)  , 

for  any  positive  integer  K,  and  for  all  times  sx  ,  tx  ,  s2  ,  t2  ,  ...  ,  sK  ,  tK 

satisfying  Tx  <  sx  <  tx  <  s2  <  t2  <  •  •  *  <  sK  <  tK  <  T2: 

K  K 

(141)  2  h >  sk  ,  h )  >  r-  2  {tk  -  sk)  -  Gx  -  K-G , 

k- 1  k- 1 

Suppose  that 

(142)  \H{x)  +  1  \-{G2  +  1)  <  W{x,  h)  <  oo  for  1  <  h  <  H{x) 

It  follows  that  property  (143)  holds  for  each  hop  h  of  x  in  the  range 

0  <  h  <  H{x)  : 

(143)  If  K  is  any  positive  integer, 

and  if  sx  ,  tx  ,  s2  .  t2  ,  ...  .  ,  tK  are  any  times 

satisfying  Tx  <  Sj  <  tx  <  s2  <  t2  <  •  •  •  <  sK  <  tK  <  T<  . 

K 

and  if  B(x,  h  +  1.  r)  <  U'(x.  h  +  \)  for  all  r  in  jj  '  .  t  h-* :i 

<  - 1 

K  K 

Y,  sk  •  tk)  >  r-  V  (tk  —  sk)  —  (h  +  l)*Cj  —  1  +  1  -c  ,  +  h 

k- 1  k-  1 


Proof  of  Lemma  4 


The  proof  is  by  forward  induction  on  h.  The  base  case  (i.e.,  h  =  0)  follows 
from  assumption  (141)  and  the  fact  that  P(x,  0,  sk  ,  tk)  =  C(x,  0,  sk  ,  tk) 
during  intervals  when  B(x,  1,  r)  <  W(x,  l)  (since  buffer  0  is  never  empty). 
For  the  induction  step,  fix  a  hop  ii  of  i  in  the  range  1  <  h  <  H(x)  . 
Property  (143)  is  assumed  to  hold  for  hop  h—  1  ,  and  it  will  be  shown  to  hold 
for  hop  h.  Let  K  be  any  positive  integer,  and  let  ,  tl  ,  s2  ,  h  >  —  ?  sk  • 
be  any  times  such  that 

(144)  Tk  <  <  tl  <  So  <  t2  <  •  •  *  <  sK  <  tK  <  T2 

and 

K 

(145)  B(x,  A  +  l,  r)  <  W(x,  h+ 1)  for  all  r  in  y  [sk  ,  tk) 

k-  1 

The  goal  is  to  show  that 


(146) 

'  K '  K 

£  h,  sk  i  lk)  >  r’  £  ih  —  sk )  —  {h  +  l)*Gi  —  A'-[(A  +  1)'C2  +  Aj 


To  prove  (146),  Lemma  3  will  be  used,  with: 

e'  =  h  •Gl  e "  =  C, 

f’  =  h-G:  +  h  -  1  /"  =  G\ 


First,  note  that  the  induction  hypothesis  can  be  rephrased  in  terms  of  time 
variables  v[  and  u(  as  follows: 


-  91  - 


(147)  If  J(l),  J( 2),  ...  ,  J(K)  are  any  positive  integers,  and  if 

.W"’-2 . W  .«?  • 

,  u'W-1  ,  vfl2^1  ,  )  —  1  Vo  ,  Uo  ,  ...  , 

V  »  UK  *  UAT  »  UAT  . VK  ’  UK 

is  any  nondecreasing  sequence  of  times  in  [Tt  ,  To)  , 

A'  /(*) 

and  if  B(x,  A,  r)  <  lV(x,  A)  for  all  r  in  (j  y  \vj.  .  u'  ')  .  the: 

k-  1  1 

K  J(k)  .  K  J(k)  . 

E  E  P{x,h-l,vl  ,ui~x)  >  r-  E  EK"1  -  fi)  -  A-C! 

/t—  1  ;-l  *-l  1 


-  E  J(k)  '(A-Co  +  A  -  1 
*- 1 


Using  (147)  and  (144),  it  is  straightforward  to  verify  condition  (107)  of  Lemma 
3.  Now  condition  (108)  of  Lemma  3  will  be  verified.  .As  in  (10S),  suppose,  for 
k  =  1,  2,  ...  ,  K  ,  that  J(k)  is  some  positive  integer.  that 

1  ,  vj^k^~l  ,  ...  ,  ,  v*1  are  some  times  satisfying 

5*  <  <  *,/(*)- 1  <  *  *  *  <  u£  <  <  tk  ,  and  that 

/(*) 

B(x,  A,  r)  >  0  for  all  r  in  y  [u^  ,  u^)  .  By  (145),  then,  session  x  will  accept 

every  chance  offered  to  it  by  the  round  robin  scheduler  at  hop  A  during 
K  J(k) 
i-  1  j-  1 

k  J(k )  a  yf.t) 

(148)  E  E  A,  .  t-')  =  E  ECV- A. 

k  —  l  ;  —  1  1  ;  —  1 


If  follows  from  assumption  (141)  that 


(H9)  V  E  p(*< h-  "i .  «4)  >  E  E  (W  ~  «i)  -  Gj  -  s  /(*)  -G 

k-  1  J-  1  *-  1  j-  1  /c-  I 

This  verifies  condition  (108).  Condition  (109)  of  Lemma  3  is  satisfied  by 
assumption  (142).  All  the  conditions  of  Lemma  3  have  been  verified. 
Conclusion  (111)  of  Lemma  3  gives  the  desired  result  (146).  This  completes  the 
proof  of  Lemma  4. 
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4. 2. 3.3  Lemma  5:  Lower  Bound  on  Throughput  of  Downstream  Subpath 

Suppose  a  system  has  been  specified  that  satisfies  the  assumptions  of 
Chapter  2  and  uses  round  robin  scheduling.  Let  x  be  some  session.  Let  Tx 
and  To  be  times  satisfying  0  <  Tx  <  T2  <  oo  .  (Note  that  T2  is  permitted 
to  be  infinite.)  Suppose  there  exist  real  numbers  r,  Gx  and  G2  such  that  the 
following  inequality  holds  for  every  hop  h  of  x  in  the  range  0  <  h  <  H(x)  , 
for  any  positive  integer  K ,  and  for  all  times  sx  ,  tx  ,  s2  ,  t2  ,  ...  ,  ,  tK 

satisfying  T1<sl<t1<s2<t2<  •  •  •  <  <  tK  <  T2  : 


(150) 


K  K 

£  C{x,  h,  sk  ,  tk)  >  r*  £  {tk  ~  sk)  -  Gx  -  K-Go 

k-l  k- 1 


Suppose  that 


(151)  [H{x)  +  1]*(G2  +  1)  <  W{x,  h)  <  CO  for  1  <  h  <  H{x) 

It  follows  that  property  (152)  holds  for  each  hop  ho  f  x  in  the  range 
0  <  h  <  H(x)  : 


(152) 


If  K  is  any  positive  integer, 

and  if  sx  ,  tx  ,  s2  ,  t2  ,  ...  ,  s %  ,  tfc  are  any  times 

satisfying  Tl<sl<tl<s2<t2<  *  *  *  <  sA-  <  <A-  <  7L 

A' 

and  if  £(x,  h,r)  >  0  for  all  r  in  ,  £*)  ,  then 

k-  1 

K  K 

£  P(x,  h,  sk  ,  tk)  >  r-  £  (tt  -  $*)  -  [tf(x)  —  h  +  \}-Gl 
k- 1  *-l 


-  A'-  [[tf(x)  -  A  +  1]-G2  +  tf(x)  -  A 


/  _  -V  .*  .*  * 


Proof  of  Lemma  5 


The  proof  is  by  backward  induction  on  A.  The  base  case  (i.e.,  A  =  H(x )) 
follows  from  assumption  (150)  and  the  fact  that 
P{x,  H(x),  sk  ,  tk)  —  C(x,  H(x),  sk  ,  tk)  during  intervals  when 

B(x,  H(x),  t)  >0  (since  buffer  if(x)+l  is  never  full).  For  the  induction 
step,  fix  a  hop  A  of  x  in  the  range 

(153)  1  <  A  <  H{x) 

Property  (152)  is  assumed  to  hold  for  hop  A,  and  it  will  be  shown  to  hold  for 
hop  A  — 1  .  Let  K  be  any  positive  integer,  and  let  sx  ,  tl  ,  s2  »  h  ,  ...  ,  sK  ,  tK 
be  any  times  such  that 

(154)  7^  <  s\  <  *i  5:  52  5:  h  <  ’  '  *  5:  SK  <  <  ^2 

and 

K 

(155)  B{x ,  A  — 1,  r)  >  0  for  all  rin  (j  [s*  ,  tk) 

k-  1 

The  goal  is  to  show  that 

K  K 

(156)  £  P(x,  A  — 1,  sk  ,  tk)  >  r-  £  {tk  -  sk)  -  [H{. r)  -  A  +  2 }‘Gl 

k-  1  k- 1 

-  A'-  [[//(x)  -  A  +  2] 'Go  +  #(x)  -  A  +  1 
To  prove  (156),  Lemma  3  will  be  used,  with: 


e'  =  Gl 

f  =  G2 


«"  =  [H{x]  -  h  +  l\-Gx 

/"  =  [tf(x)  —  h  +  l]*G2  +  H{x)  -  h 


First,  condition  (107)  of  Lemma  3  will  be  verified.  As  in  (107),  suppose,  for 

k  —  1,  2,  ...  ,  K  ,  that  J(k)  is  some  positive  integer,  that 

v 1  ,  2  ,  ...  ,  v*1  ,  u°  are  some  times  satisfying 

sk  <  vk ^  5:  u/^-1  <  vk^~1  <  «/^“2  <  •  *  •  <  <  ut°  <  tk  ,  and 

/(k)  .  .  , 

that  £(x,  A,  r)  <  VF(x,  A)  for  all  r  in  y  [vj£  ,  *)  .  By  (155),  then,  session 

j-i 

x  will  accept  every  chance  offered  to  it  by  the  round  robin  scheduler  at  hop 
K  J(k)  .  . 

h-1  during  \j  |J  {y{  ,  *]  : 

i- 1  /- 1 

K  J(k)  .  K  J{k )  .  , 

(157)  £  £P(x,h-l,vt,url)  -  S 

k-  1  1  t-  1  j-  1 

If  follows  from  assumption  (150)  that 


(158) 

k  j(k)  .  ,  a:  /(*)  .  , 

E  E  p(x»  fc-i.  ,  ui~  )  >  r'  E  EW 


-  E  At)  *g2 


This  verifies  condition  (107).  Next,  note  that  the  induction  hypothesis  can  be 
rephrased  in  terms  of  time  variables  v£  and  v£  as  follows: 


XA3  *1  V  *T« 5 *  VW  VWV*  VTU*  l/V  TOf  'JVWWi 
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(159)  If  /(l),  J(2),  ...  ,  J(K)  are  any  positive  integers,  and  if 


«'<■>  x<*>  ,«'<*>-' , 


.  “i1 .  W  , 


«a^i ,  .2'<!> ,  «2y<2>-‘ ,  . »2> ,  vi  , ... , 

«£*>  ,  v#*>  ,  a/?*1"1  ,  Of?*’-1 . «L 

is  any  nondecreasing  sequence  of  times  in  [Tx  ,  T2)  , 

K  J[k)  . 

and  if  B(x,  h,  r)  >  0  for  all  rin  (j  [u£  ,  v{)  ,  then 

*-i y- l 

K  J(k)  K  J(k)  . 

E  E  p(x »  A’  *i  I  vi)  >  r*  E  E  (W  -  ui)  -  [^(x)  “  h  + 

*-l j-1  *-l  1 


K 

E  J(k) 

k- 1 


\[H(x)  -  A  +  1]-G2  +  tf(x)  - 


Using  (159)  and  (154),  it  is  straightforward  to  verify  condition  (108)  of  Lemma 
3.  Condition  (109)  of  Lemma  3  is  satisfied  by  assumptions  (151)  and  (153).  All 
the  conditions  of  Lemma  3  have  been  verified.  Conclusion  (110)  of  Lemma  3 
gives  the  desired  result  (156).  This  completes  the  proof  of  Lemma  5. 


V 


4. 2. 3.4  Lemma  6:  Lower  Bound  on  Throughput,  given  Lower  Bound  on 
Chances 


Suppose  a  system  has  been  specified  that  satisfies  the  assumptions  of 
Chapter  2  and  uses  round  robin  scheduling.  Let  x  be  some  session.  Let  Tx 
and  T2  be  times  satisfying  0<Tl<T2<oo.  (Note  that  T2  is  permitted 
to  be  infinite.)  Suppose  there  exist  real  numbers,  r,  Gx  and  G2  such  that  the 
following  inequality  holds  for  every  hop  h  of  x  in  the  range  0  <  h  <  H( x)  , 
for  any  positive  integer  K,  and  for  all  times  Sj  ,  tx  ,  s2  ,  t2  ,  ...  ,  sK  ,  t % 
satisfying  <  s1  <  tx  <  s2  <  t2  <  *  •  *  <  <  T2  : 


(160) 


K  K 

£  C(x,  h,  sk  ,  tk)  >  r*  Y  (tk  -  sk )  —  Gl  —  K-G2 

k— 1  k-l 


Suppose  that 


(161)  [H(x)  +  1]-(C?2  +  1)  <  W{x,  h)  <  oo  for  1  <  h  <  H{x) 

It  follows  that,  for  each  hop  A  of  a:  in  the  range  0  <  h  <  H(x )  ,  for  any 

positive  integer  K,  and  for  any  times  ,  s2  ,  t2  ,  ...  ,  sK  ,  tK  satisfying 

Ti  <  Si  <  tx  <  So  <  t2  <  •  •  •  <  sk  <  tK  <  T2  : 


(162)  YP(*’h’*t  ,tk) 

k-  1 


>  r-  Y  (<k  ~  **)  -  Wx)  +  M-Gi  -  K-  fjff(x)  +  l]’G,  +  H(x) ] 


<• 

I* 

I 


4 


a 
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Proof  of  Lemma  6 


Let  h  be  any  hop  of  x  in  the  range  0  <  A  <  H(x)  .  Let  K  be  any  positive 
integer,  and  let  sl  ,  t1  ,  s2  ,  t2  ,  ...  ,  be  any  times  satisfying 


(163) 


T\  <  <  tx  <  s2  <  £2  <  "  *  *  <  s#  <  ^  <  T2 


t 


If  h  =  0  ,  then  (162)  follows  directly  from  Lemma  5,  since  B(x,  0,  r)  >  0  for 
all  times  r  >  0  ;  so  assume  that  h  >  1  .  To  prove  (162),  Lemma  3  will  be 


used,  with: 


f  -  h-G2  +  h  -  1 


e"  =  [J5T(x)  —  A  +  l]-a1 

/"  »  [ff(ar)  -  A  +  1]-G2  +  H{x)  -  h 


First,  note  that  Lemma  4  can  be  applied  to  hop  A  — 1  and  rephrased  in  terms 
of  time  variables  and  u{  to  yield  the  following  property: 


I 


(164)  If  /(l),  /( 2),  ...  ,  J(K)  are  any  positive  integers,  and  if 

v{M ,  up)-' ,  „'(»-■ ,  . 

,  «2f2|“  1  ,  1  .  W2(2'1-2  ,  ...  ,  Kj  ,  »2°  .  ...  , 

vji«) ,  „/m-. ,  ,  „^>-2 . 

is  any  nondecreasing  sequence  of  times  in  [Tj  ,  T2)  , 

K  J(k)  . 

and  if  5(j,  A,  r)  <  \V{x ,  A)  for  all  r  in  y  [j  )  ,  then 

*-l  1 


K  J(k)  .  ,  A'  /(*)  .  , 

E  .  «/  )  >  r*2  E  («rl-  -  A- 

:-l  j-  1  *-l  ;-l 


A'  /(*) 


-  £  7(A)  *(A*G2  +  A  -  1) 

*- 1 


Using  (164)  and  (163),  it  is  straightforward  to  verify  condition  (107)  of  Lemma 
3.  Next,  note  that  Lemma  5  can  rephrased  in  terms  of  time  variables  ul  and  vl 
to  yield  the  following  property:  . 


(165)  If  J(  1),  /( 2),  ...  ,  J(K )  are  any  positive  integers,  and  if 


/( i) 


«/(!)  „/(!)- 1  u}  vl 

i  v\  »  “i  >  v\  >  •••  »  “i  >  ui  > 


,  vP 


up-'  ,  vp-' 


.,1  1 
,  U  2  ,  Vo  , 

’  UK  ’  VK 


,  V#K)  ,  ,  „#*)- 

is  any  nondecreasing  sequence  of  times  in  [T^  ,  To)  , 

K  J{k)  . 

and  if  B(x,  h,  r)  >  0  for  all  rin  [j  y  [ u {  ,  v{.)  ,  then 

k- 1 ;-l 

K  J(k )  K  J{k)  . 

E  E  »  ui)  >  r*  E  E  W  -  ui )  -  [#(*)  -  h  +  i]*^i 

*- l j- l 


K 

E-W 


[[tf(x)~  A  +  1]-G2  +  tf(x)~  A' 


Using  (165)  and  (163),  it  is  straightforward  to  verify  condition  (108)  of  Lemma 
3.  Condition  (109)  of  Lemma  3  is  satisfied  by  assumption  (161).  All  the 
conditions  of  Lemma  3  have  been  verified.  Conclusion  (111)  of  Lemma  3  gives 
the  desired  result  (162).  This  completes  the  proof  of  Lemma  6. 


00- 


4.2.4  Lemma  7:  Upper  Bound  on  Throughput,  given  Lower  Bound  on 
Throughput 

Suppose  a  system  has  been  specified  that  satisfies  the  assumptions  of 
Chapter  2  and  uses  any  scheduling  discipline.  Suppose  each  session  has  a 
well-defined  demand  rate  Suppose  that 

(167)  W'  <  oo 

Let  x  be  some  session.  Let  h  be  some  hop  of  x  in  the  range  0  <  h  <  H(x)  . 
Let  K  be  a  positive  integer,  and  let  sx  ,  tx  ,  s2  >  *2  >  •••  *  sk  *  tK  be  times 
satisfying  0  <  <  s2  <  t2  <  *  *  •  <  sK  <  tK  .  Let  Gx  be  a  real 

number  satisfying 

(168)  £  P(x,  h,  tk  ,  sk+l)  >  Rf(I(x))‘  £  (st+1  -  tk)  -  Gx 

l  *-i 

Let  A  be  a  non-negative  real  number  such  that 

(169)  C{x,  0,  Sj  ,  tK)  <  \(x)-(tK  -  sx)  +  A 
Let 

(170)  G2>  0 

be  a  real  number  such  that,  for  every  session  y  with  I(y)  <  I{x)  and  for 
every  link  l  used  by  y , 

(171)  P'(y,  1,  ,  tK)  >  RF(I(y))-(tK  -  «,)  -  C2 


It  follows  that 
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l**'. 


(172)  ^P(x,h,sk  ,tk) 
jfc-l 

<  Rf(I{x)Y  E  (**  -  sk)  +  {N  -  1  yc2  +  Gx  +  W-H  +  A 

i—  1 

Proof  of  Lemma  7 

From  assumption  (168),  it  follows  that 

K  K- 1 

E  P{x,  h ,  sk  ,  tk )  =  P^x,  h,  Si  ,  tK)  E  P{x i  h i  tk  »  sk+  i) 

*-i  i-l 

(173)  <  P(x,  h,slf  tK)  -  Rf(I(x)Y  £  (s*+i  “  h)  +  Gi 

Jt-i 

Now  the  term  P(x,  h,  s1  ,  tK)  will  be  bounded.  Recall  that,  by  the  properties 
of  the  max-min  flow  criterion  (Section  4.1),  every  session  has  at  least  one 
bottleneck  hop.  Let  h*  be  any  bottleneck  hop  of  x,  0  <  h*  <  H{ x)  ,  and 
consider  the  following  claim: 

(174)  P(x,  h,  sl  ,  tK)  <  P(x,  k*,  8X  ,  tK)  +  W’'H 

If  h  —  h*  ,  then  (174)  is  obviously  true.  If  h  >  h*  ,  then  (174)  is  true 

because  P(x.  h,  s{  ,  tK)  can  be  no  more  than  P(x ,  h*,  .  tF)  plus  the  total 

number  of  packets  present  at  time  Sj  in  all  the  buffers  between  hops  h*  and 
h  ;  there  are  at  most  H  such  buffers,  each  of  capacity  at  most  lb'  .  If 

h  <  h*  ,  then  (174)  is  true  because  P(x,  h,  s{  ,  tK)  can  be  no  more  than 

P(x,  h*,  sl  ,  tK)  plus  the  total  amount  of  spare  capacity  at  time  Sj  in  all  the 
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buffers  between  hops  h  and  h*  ;  there  are  at  most  H  such  buffers,  each  of 
capacity  at  most  W  .  This  proves  (174).  Inequalities  (173)  and  (174)  show 
that 

(175)  £p(x,  h,sk,tk) 

*-i 

<  P(x,  h*,  Sl  ,  tK)  -  RF(I(x)yKZ(sk+l  -  tk)  +  +  W'-H 

k-  1 

Now  P(x,  h* ,  sx  ,  tft)  will  be  analyzed,  using  the  properties  of  bottleneck 
hops.  There  are  two  cases  to  consider.  If  h*  —  0  ,  it  follows  from  assumption 
(169)  and  definition  (65)  that 

P(x,  h*f  ,  tK)  =  P(x,  0,  ,  tK) 

<  C(x ,  0,  Sj  ,  tK) 

<  X(l  )'{tK  -  Sy)  +  A 

(176)  =  RF(I(x))-(tK  -  Sj)  +  A 

If  1  <  h*  <  H(x )  ,  let  l  denote  the  link  corresponding  to  hop  h* ,  and  let  Y 
denote  the  set  of  sessions  y  #  x  that  use  l.  Note  that,  by  definition  (64). 
I(y)  <  I(x)  for  all  sessions  y  in  }'.  Obviously,  x  can  only  use  slots  in  (.Cj  .  tF] 


that  are  not  used  bv  sessions  in  Y: 


P(x,  h*,  ,  tK)  —  P'(x,  /,  5t  ,  tK) 

(177)  <  to-*i)  -  Y:p,(y>l’si  >tK) 

yer 

Applying  assumptions  (171)  and  (170)  and  definition  (64)  yields: 

P(x,  h*,  sx  ,  tK)  <  (tK  -  sj  -  £  f RF{I{y))m{tK  -  sj  -  Go 

y€  YL 


1-  £RF(I(y)) 

y€Y 


*i) 


+  \  y\-g2 


< 


1 


E  Wiv)) 

yeY 


si ) 


+  (A'  -  i)-C2 


(178)  -  RF(I(x))‘(tK  -  Sl)  +  (N  -  l)*G2 


Since  A  and  G2  are  non-negative,  inequalities  (176)  (for  the  case  where 
h*  —  0  )  and  (178)  (for  the  case  where  h*  >  0  )  may  be  combined  into  a 
single  inequality: 

(179)  P( x,  h*,  Sj  ,  tK)  <  RF(I(x))-(tK  —  Sj)  +  A  +  (;V  —  1)*G2 


Substituting  (179)  into  (175)  gives  the  desired  result  (172): 


05‘ - 


4.3  Transient  Analysis  of  Smooth  Demand  Case 

This  section  contains  a  single  result,  Theorem  2.  The  theorem  analyzes  a 
system  during  an  interval  {Tl  ,  T2 )  of  smooth  demand.  Specifically,  it  is 
assumed  that  there  exists  a  constant  A  such  that  the  demand  of  each  session  x 
over  each  subinterval  (s,  t]  of  (T^  ,  T2)  is  within  A  packets  of  the  nominal 
amount  X(x)*(f— s)  .  Most  window  sizes  are  assumed  to  be  at  least 
Z'(H+  1)S'NS~  1,(A+2)  .  Theorem  2  concludes  that  the  throughput  of  each 
session  x  at  each  hop  over  each  subinterval  (s,  t]  of  (T1  ,  T2)  is  within 
(i/’+l)5,A^25_1-(Vr,+  3A+4)  packets  of  the  fair  amount  RF (7(i))’(£  —  s)  . 
Note  that  this  unfairness  bound  increases  with  the  maximum  window  size  \V  . 
This  is  not  surprising,  since  the  system  should  go  through  a  transient  period 
during  which  buffers  upstream  of  bottleneck  hops  fill  and  buffers  downstream 
of  bottleneck  hops  drain.  One  would  not  expect  to  see  fair  flows  until  the 
buffers  levels  stabilize.  Obviously,  the  transient  can  be  more  pronounced  if  the 
windows  are  larger. 

Let  us  outline  the  proof  of  Theorem  2.  The  proof  is  by  induction  on  the 
congestion  index  i  of  a  session.  The  induction  hypothesis  gives  upper  and 
lower  bounds  on  the  throughput  of  each  session  with  congestion  index  less  than 
i.  Recall  that  the  theorem  assumes  upper  and  lower  bounds  on  the  demand  of 
each  session.  The  proof  of  the  induction  step  has  three  parts.  First,  Lemma  2 
uses  the  upper  bound  on  throughput  from  the  induction  hypothesis  plus  the 
properties  of  round  robin  scheduling  to  deduce  a  lower  bound  on  the  number 
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of  chances  offered  to  each  session  with  congestion  index  t  at  each  link.  Then, 
Lemma  6  uses  this  derived  lower  bound  on  chances,  the  given  lower  bound  on 
demand,  and  the  assumption  of  large  windows  to  deduce  a  lower  bound  on  the 
throughput  of  each  session  with  congestion  index  i.  Finally,  Lemma  7  uses  this 
derived  lower  bound  on  throughput,  the  lower  bound  on  throughput  from  the 
induction  hypothesis,  the  given  upper  bound  on  demand,  and  the  properties  of 
max-min  fairness  (viz.,  the  existence  of  bottleneck  hops)  to  deduce  an  upper 
bound  on  the  throughput  of  each  session  with  congestion  index  j.  This 
preview  should  make  the  proof  a  little  easier  to  follow. 
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4.3.1  Theorem  2:  Throughput  Bounds 

Suppose  a  system  has  been  specified  that  satisfies  the  assumptions  of 
Chapter  2  and  uses  round  robin  scheduling.  Suppose  each  session  x  has  a 
well-defined,  real  demand  rate  X(x)  ,  0  <  X(x)  <  1  .  Let  Tl  and  To  be 
times  satisfying  0<T1<T2<oo.  (Note  that  T2  is  permitted  to  be 
infinite.)  Suppose  there  exists  a  nonnegative  real  number  A  such  that,  for  each 
session  x  and  for  all  times  s  and  t  satisfying  Tx  <  s  <  t  <  T2  , 

(ISO)  |  C{x,  0,  s,  t)  -  X(x)*(t  -  s)  |  <  A 

Suppose  that,  for  each  session  x, 

(181)  3m(H+l)s 'Ns~  1*(A+2)  <  W{x,h)  <  oo  for  1  <  h  <  H{x) 

It  follows  that,  for  each  session  x,  for  each  hop  h  of  x  in  the  range 
0  <  A  <  H(x )  ,  and  for  all  times  s  and  t  satisfying  <  s  <  t  <  T2  , 

(182)  \P{x,h,s,t)  —  RF(I(x))'(t  —  s)  |  <  (H+l)S'N2S-liW+ZA  +  4) 
Proof  of  Theorem  2 

In  order  to  show  (182),  properties  (183)  -  (185)  will  be  proved. 

(183)  For  each  session  x,  for  each  hop  h  of  x  in  the  range  0  <  h  <  H(x)  . 

for  any  positive  integer  A',  and  for  any  times  Sj  ,  tl  .  s.2  ,  t.2 . sK  . 

satisfying  T±  <  st  <  tx  <  s2  <  t2  <  •  •  •  <  sK  <  tK  <  To  : 

K  K 

E  c(x'  A’  sk  .  h)  >  ^c(x>  hY  E  (h  -  sk)  -  Dcl[X  /(x),  70 

*-l  *-l 


»  y  r  »  ~  •  '  * 


.'••.A/ 


s  s 


(184)  For  each  session  x,  for  each  hop  h  of  x  in  the  range  0  <  h  <  H(x)  , 

for  any  positive  integer  K,  and  for  any  times  s1  ,  tx  ,  s2  »  4  >  •••  *  SK  > 

satisfying  <  4  <  s2  <  <  *  '  *  ^  SK  ^  tfC  <  T 2  : 

K  K 

E  P{x,  h,  sk  ,  4)  >  £/•(/(*))*  E  (4  -  s*)  -  DPL{At  I(x),  K) 

At- 1  A-l 

(185)  For  each  session  x,  for  each  hop  h  of  x  in  the  range  0  <  h  <  #(x)  , 

for  any  positive  integer  K,  and  for  any  times  sx  ,  ,  s2  ,  t2  ,  ...  ,  sK  ,  tK 

satisfying  Tx  <  $!  <  tx  <  s2  <  t2  <  *  *  *  <  sK  <  tK  <  T2  : 

K  K 

E  p(x>  5*  *  4)  <  #fU(*))*  E  (4  -  -sa)  +  /(*),  AT) 

*-l  A:—  1 

The  proof  is  by  induction  on  the  congestion  index  I(x)  of  the  session  x. 
Contrary  to  custom,  the  induction  step  will  be  proved  before  the  base  case  is 
addressed.  Fix  a  congestion  index  t  >  1  .  The  induction  hypothesis  asserts 
that  (1S3),  (184),  and  (185)  hold  for  all  sessions  x  with  I(x)  <  i  .  It  must  be 
shown  that  (183),  (184),  and  (185)  hold  for  all  sessions  x  with  /(x)  =*  i  . 

First  consider  (183).  Let  x  be  any  session  with  /(x)  =  :  .  Let  h  be  any 
hop  of  x  in  the  range  0  <  h  <  H(x)  .  Let  K  be  any  positive  integer, 
and  let  ,  tx  ,  s2  ,  t2  ,  .  .  .  ,  be  any  times  satisfying 

T !  <  .$j  <  4  <  s2  <  t2  <  •  *  •  <  sA-  <  tfc  <  T-2  •  If  hop  h  =  0  .  it  follows 

from  assumption  (ISO),  definition  (68),  and  conclusion  (00)  of  Lemma  1  that 


yy-yrj I-J  ’-V  »V  ■  .»v»L»VVV  ir*  V  CH  C* '.%'  1% uV’^.’V'.V.*.- T7 W  V-y.^X'TH.T^'TT.’^.TJT^THITS.^T.^'lT.l^T^Tq 
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K  K 

£  C(x,  0,  Sk  ,  tk )  >  \{x )•  ^  ( tk  -  sk)  -  K- A 
Jfc-1  l  —  1 

=  Rc(x,  0 )•  J]  -  sk )  -  fiT-A 
*- 1 

(186)  >  J2c(x,  0)*  £  (4  -  5*)  -  Dcl( A,  t,  AT) 

*- 1 

If  1  <  A  <  ^f(x)  ,  Lemma  2  will  be  used,  with  G  —  Dpi 7(A,  *  —  1 ,  K )  .  By- 
conclusion  (84)  of  Lemma  1,  condition  (98)  of  Lemma  2  holds.  By  the  induction 
hypothesis,  (185)  holds  for  all  sessions  with  congestion  index  less  than  t.  This 
fact,  along  with  conclusion  (88)  of  Lemma  1,  verifies  condition  (99)  of  Lemma  2. 
From  conclusion  (100)  of  Lemma  2  and  conclusion  (92)  of  Lemma  1,  it  follows 
that 

K  K 

£  c(x .  h’  sk  ,h)  >  Rcix’  hY  £  (ijfc  -  sk )  -  {N-l)-DPU(A,  i-1,  fv)  -  /v 


4- 1 

Jfc-1 

(187) 

>  *c(*.  *)'  E  (‘4  -  *4)  -  Dcl(&,  i,  K) 

Jfc-1 


This  completes  the  proof  of  (183)  for  the  induction  step. 

Now  (1S4)  will  be  proved.  Let  x  be  any  session  with  I(x)  =  i  .  Lemma  6 
will  be  used,  with  r  =  Rp(i)  .  G {  =  Ec '(A.  i)  and  GL  =  F^(A.  i)  .  To 
verify  condition  (160)  of  Lemma  6.  let  h  be  any  hop  of  x  in  the  range 

0  <  A  <  H{x)  ,  let  K  be  any  positive  integer,  and  let 
,  <[  ,  5j  ,  f2 . ,  t ft  be  any  times  satisfying 


\ 


1 


just  proved  for  all  sessions  with  congestion  index  i.  Applying  (183),  (70),  and 
definition  (78)  verifies  condition  (160): 

K  K 

E  c(x’  h ’  sk  .  h)  >  Rcix>  hY  E  (*4  -  sk )  -  DCL(A,  *.  K) 

4-1  4-1 


>  *f(*>  E  (4  -  «*)  -  ^>cl(A,  «,  K ) 

4-  1 


(188) 


=  £/•(*)*  E  (*4  ”  si)  ~  0  —  K'RCl(^’  0 

4-1 


Condition  (161)  of  Lemma  6  holds  because  of  assumption  (181)  and  definition 
(74).  Now  conclusion  (162)  of  Lemma  6,  definition  (78),  and  conclusion  (94)  of 
Lemma  1  can  be  applied  to  show  that,  for  each  hop  h  of  x 

in  the  range  0  <  h  <  H(x)  ,  for  any  positive  integer  K, 

and  for  any  times  sx  ,  tx  ,  s2  ,  to  satisfying 

T.  <  <  So  <  <  *  •  *  <  <  tK  <  To  : 


£P(x,  k,sk  ,  tk) 

k-  1 

>  WY  E  («*  -  •*)  -  (^  +  0  -  K‘[(H  +  1)*Fcl(A,  0  +  H] 

k-  1 


*#•(•)*  E  (h  -  sk)  -(H+  l)'DCi(A,  i,  K)  -  K'H 

k-  1 


>  WYE  (tk  -  «*)  -  DPL(A,i,K) 

k- 1 


This  completes  the  proof  of  (184)  for  the  induction  step. 

Next  (185)  will  be  proved.  Let  x  be  any  session  with  I(x)  =  i  .  Let  h  be 
any  hop  of  x  in  the  range  0  <  A  <  H[x )  .  Let  K  be  any  positive  integer, 
and  let  sx  ,  tx  ,  s2  ,  f2  ,  .  .  .  ,  ,  tK  be  any  times  satisfying 

T\  <  <  ti  <  s2  <  ^2  <  •  •  •  <  5^-  <  t%  <  T2  ■  Lemma  7  will  be  used, 
with  Gx  —  Dpl{ A,  *,  K—  1)  and  G2  =  DPi{A ,  t,  1)  .  Condition  (167)  of 
Lemma  7  follows  from  assumption  (181).  If  K  =  1  ,  condition  (168)  of  Lemma 
7  holds  because  of  conclusion  (82)  of  Lemma  1.  If  K  >  1  ,  condition  (16S) 
holds  because  (184)  was  just  proved  for  all  sessions  with  congestion  index  i. 
Condition  (169)  of  Lemma  7  holds  because  of  assumption  (ISO).  Condition 

(170)  of  Lemma  7  holds  because  of  conclusion  (82)  of  Lemma  1.  By  the 
induction  hypothesis.  (184)  holds  for  ail  sessions  with  congestion  index  less 
than  t .  This  fact,  along  with  conclusion  (86)  of  Lemma  1,  verifies  condition 

(171)  of  Lemma  7  for  those  sessions  y  with  I{y)  <  i  .  For  those  sessions  y 
with  I(y)  —  i  ,  condition  (171)  holds  because  (184)  was  just  proved  for  all 


sessions  with  congestion  index  t.  From  conclusion  (172)  of  Lemma  7  and 
conclusion  (97)  of  Lemma  1,  it  follows  that 

K  K 

E  P(z,  h .  -  h)  <  RF{'Y  E  (h  ~  «*)  +(N-  1)‘Dpl(A,  i,  1) 

k- 1  i-1 

4-  £)/>£_( A,  i,  K—l)  +  W'H  +  A 
(189)  <  Rp(t)'  E)  (4  —  s±)  +  Dpu(A,  i,  K ) 

k-  l 

This  proves  (185),  completing  the  induction  step. 

The  proof  for  the  base  case  (i.e.,  i  =  1)  is  identical  to  the  induction  step, 
considering  the  following  point.  In  the  induction  step,  the  induction 
hypothesis  was  invoked  to  verify  the  assumptions  of  Lemmas  2  and  7  for 
sessions  y  with  I(y )  <  i  .  For  the  base  case,  there  are  no  such  sessions  y,  so 
verifying  these  assumptions  is  trivial. 

This  completes  the  proofs  of  (183),  (184),  and  (185).  Conclusion  (182) 
follows  from  results  (184)  -  (185),  definitions  (71)  -  (80),  and  the  fact  that 
I(x)  <  5  for  all  sessions  x. 


This  completes  the  proof  of  Theorem  2. 


4.4  Steady-State  Analysis  of  Smooth  Demand  Case 


This  section  examines  the  steady-state  behavior  of  systems  with  smooth 
demand.  Specifically,  it  is  assumed  that  there  exists  a  constant  A  such  that 
the  demand  of  each  session  x  over  each  interval  (s,  £]  is  within  A  packets  of 
the  nominal  amount  X(z )m(t—s)  .  Most  window  sizes  are  assumed  to  be  at 
least  3’(H+l)S ’Ns~l'(&+2)  .  Corollary  1  of  Theorem  2  concludes  that  the 
long-term  average  throughput  Ra{x)  of  each  session  x  equals  its  fair  rate 
Rp(I(x))  .  In  other  words,  smooth  demand  and  large  windows  are  sufficient  for 
throughput  fairness.  Example  3  shows  that  large  windows  are  sometimes 
necessary  as  well,  and  that  throughputs  can  be  very  unfair  if  the  windows  are 
too  small.  This  example  consists  of  eight  links  and  2 N+ 1  sessions,  where  N 
can  be  any  even  integer  greater  than  four.  The  demand  is  perfectly  smooth 
(A  =  0),  but  the  window  size  W(x,  2)  for  buffer  2  of  a  particular  session  x  is 
less  than  VzN  .  Because  of  this  inadequate  window  size  and  an  unfortunate 
choice  of  round  robin  rings  and  initial  ring  positions,  the  long-term  average 


throughput  of  x  is  unfair  by  a  factor  of 


N 

2*  W(x,  2)  ' 


This  section  also  presents  a  steady-state  analog  of  Theorem  2.  Theorem  3 
states  that  there  exists  a  time  Tss  >  0  such  that  the  throughput  of  each 
session  x  at  each  hop  over  each  interval  (s,  t]  later  than  Tss  is  within 
(H+  l)S’NS~  1,(A+2)  packets  of  the  fair  amount  RF(I(x))"(t  —  s)  and  such 
that  a  similar  lower  bound  applies  to  chances.  (For  concreteness.  T$s  is 
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defined  to  be  the  earliest  such  time.)  Note  that  the  bound  on  throughput 
unfairness  in  steady  state  is  tighter  than  the  transient  bound  of  Theorem  2. 
Moreover,  the  steady-state  bound  does  not  depend  on  the  window  sizes  (except 
for  the  assumption  that  the  windows  are  large  enough). 

The  proof  of  Theorem  3  is  similar  to  that  of  Theorem  2:  the  proof  of 
Theorem  3  is  also  by  induction  on  the  congestion  index,  and  the  proof  of  the 
induction  step  also  invokes  Lemmas  2,  6  and  7  to  generate,  respectively,  a 
lower  bound  on  chances,  a  lower  bound  on  throughput,  and  an  upper  bound  on 
throughput  for  all  sessions  with  a  particular  congestion  index.  The  derived 
throughput  bounds  are  of  the  form 

(190)  -  /'  <  P(z,  h,  s,  t)  -  RF(I{x))-{t  -  s)  <  J" 

where  f"  is  a  function  of  the  maximum  window  size  W  (and  also  of  A.  /(x), 
H ,  and  N),  while  /'  does  not  depend  on  W'  ,  and  0  </'</".  At  this 
point  Theorem  3  invokes  Lemma  8  of  Appendix  A.l  to  conclude  that  in  steady 
state,  i.e.,  for  sufficiently  large  s  and  t, 

(191)  -  /'  <  P(x,  h,  s,  t)  -  RF(I{x))-(t  -  s)  <•/'+  1 

In  this  way  Theorem  3  derives  bounds  on  the  throughput  unfairness  in  steady 
state  that  are  tighter  than  the  transient  bounds  cf  Theorem  2  and  are 
independent  of  W  . 

Included  in  this  section  are  four  corollaries  of  Theorem  3  about  the  steady- 
state  buffer  levels.  In  terms  of  the  ti  me  Tss  (when  the  steady-state 
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throughput  bounds  take  effect),  let  us  define  rn(x,  h)  and  M(x,  h )  for  each 
buffer  h  of  each  session  x  such  that  1  <  h  <  /f(i)+l  : 

(192)  m(x,  h)  =  min  B(x,  h,  t) 

Tss 

(193)  M(x,  h )  =  max  B(x,  h,  t ) 

t  >  Tss 

Since  B(x,  0,  t)  —  oo  for  all  times  t  >  0  ,  we  also  define 

(194)  m(x,  0)  ==  M(x,  0)  =  oo 

Corollary  2  gives  an  upper  bound  on  the  range  M(x,  h )  —  m(x,  h)  of  a  buffer 
level  after  time  Tss  the  bound  does  not  depend  on  the  window  sizes  (except 
for  the  assumption  that  the  windows  are  large  enough).  Corollary  3  proves 
that,  after  time  Tss  ,  buffers  that  are  slightly  upstream  of  bottleneck  hops  are 
sometimes  full  and  are  never  empty,  while  buffers  that  are  slightly  downstream 
of  bottleneck  hops  are  sometimes  empty  and  are  never  full.  A  bottleneck  hop 
A*  of  a  session  x  in  the  range  0  <  h*  <  H(x )  is  called  a  pure  bottleneck  for  x 
if  there  exists  a  time  T  >  0  such  that  all  session  j’s  chances  at  hop  h*  after 
time  T  are  successful: 

(195)  C(x,  h*,  t—  1.  t)  =  P(x ,  h*,  t—  1.  t )  for  all  times  t  >  T 

In  other  words,  the  window  mechanism  does  not  impede  the  flow  for  a  session 
at  a  pure  bottleneck  hop;  packets  and  permits  are  always  available  whenever  a 
chance  for  transmission  arises.  Corollary  4  concludes  that  every  session  has  at 
least  one  pure  bottleneck  hop.  Example  4  shows  that  impure  bottlenecks  do 


4.4.1  Corollary  1:  Fairness  of  Average  Throughputs 


Suppose  a  system  has  been  specified  that  satisfies  the  assumptions  of 
Chapter  2  and  uses  round  robin  scheduling.  Suppose  each  session  x  has  a 
well-defined,  real  demand  rate  \(x)  ,  0  <  \(x)  <  1  .  Suppose  there  exists  a 
nonnegative  real  number  A  such  that,  for  each  session  x  and  for  all  times  s 
and  t  satisfying  0  <  s  <  t  , 

(196)  |  C(x ,  0,  s,  t)  —  X(x)*(£  —  5)|  <  A 
Suppose  that,  for  each  session  x, 

(197)  3-(ff+l)5\IVs“1-(A+2)  <  \V{x,h)  <  00  for  1  <  h  <  H{x) 

It  follows  that,  for  each  session  x,  the  long-term  average  throughput  ^(x) 
exists  and  equals  the  fair  rate  RF(I(x))  . 


Proof  of  Corollary  1 


Note  that  the  assumptions  of  Theorem  2  are  satisfied,  with  Tx  —  0  and 
To  —  00  .  Let  x  be  any  session.  It  follows  from  conclusion  (182)  of  Theorem  2 
that  for  all  times  t  >  0  , 


P(x.  H{ x).  0.  t) 
t 


(198) 


|  P{x,  H{x).  0.  0  -  Rp(I{x)yt  \ 
t 

<  {H  +  1)S-.V2S-^(H"  +  3A  +  A) 

~  t 


•  j  .• 

r'«  ^ 


4.4.2  Example  3:  Unfairness  with  Small  Windows 

Consider  a  system  that  satisfies  the  assumptions  of  Chapter  2  and  has  the 
layout  shown  in  Figure  8.  The  network  contains  links  lxl  ,  ll2  ,  /2  l  *  and 
/2(2  •  (For  each  of  these  links,  there  is  another  link  with  opposite  direction  that 
is  not  shown  in  Figure  8  and  is  used  only  to  return  flow  control  permits.)  For 
j  =  1,  2  ,  there  are  five  sessions  yjx  ,  yj  2  >  ...  ,  yj  5  that  use  lj  j  followed  by 
lj  2  ,  and  there  are  five  sessions  yj  6  ,  yj7  ,  ...  ,  yj  l0  that  use  only  lj  x  .  There 
is  also  a  session  x  that  uses  lx  2  followed  by  l2  2  .  Every  session  in  the  system 
has  heavy  demand;  i.e., 

(200)  C(x,  0,  t-1,  t )  =  C{yjik  ,  0,  t-1,  t)  =  1 

for  3  *  1,  2,  for  k  —  1,  2,  ...  ,  10,  and  for  all  times  t  >  1  .  The  max-min 
fair  rate  for  session  x  is  l/2  ,  while  the  other  sessions  deserve  rates  of  l/lO 
each.  The  window  size  for  each  buffer  A  >  1  of  each  session  is  at  least  two 
but  finite.  In  particular,  W(x,  2)  =  2  ,  which  is  smaller  than  Theorem  3 

requires.  Table  1  shows  the  buffer  levels  at  time  0.  Round  robin  link 

scheduling  is  used.  For  j  =  1,  2  ,  the  ring  at  lj  x  is  y}  j  ,  y;-  2  ,  ...  ,  y  -  10  . 

The  ring  position  of  lx  x  at  time  0  is  yx  l  ,  while  the  initial  ring  position  of 

*2,1  is  V-2, 6  •  For  j  —  1,  2  ,  the  ring  at  lj  2  is  yj  x  .  yj  2 . I/;,5  .  J*  .  The 

ring  position  of  /12  at  time  0  is  x,  while  the  initial  ring  position  of  /->.■>  is 
V2,S  ■ 


*> 


This  system  is  periodic,  with  a  period  of  ten  slots.  Table  2  shows  which 
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session  uses  each  slot  at  each  Hr  ..uring  the  interval  (0,  10]  .  During  the  first 
half  of  this  interval,  session  x  nsmits  no  packets  over  link  lx  2  (because  the 
link  is  busy  serving  other  sessions)  and  only  W(x ,  2)  =  2  packets  over  /2  2 
(because  x  runs  out  of  packets  in  buffer  2).  During  the  second  half  of  the 
interval,  x  transmits  no  packets  over  l2  2  (because  the  link  is  busy  serving 
other  sessions)  and  only  W(x,  2)  =  2  packets  over  /12  (because  x  runs  out 
of  permits  for  buffer  2).  The  long-term  average  throughput  of  session  x  is 
2/l0  ,  well  below  its  fair  rate  of  l/2  .  The  long-term  average  throughputs  of 
the  other  sessions  are  fair.  Three  tenths  of  the  capacities  of  lx  2  and  /2  2  are 
wasted. 

Using  the  same  network,  similar  examples  can  be  constructed  that  have 
different  numbers  of  sessions  and  different  window  sizes.  Let  N  be  an  even 
integer  greater  than  four.  For  j  —  1,  2  ,  there  are  VzN  sessions  using  /;1 
and  lj  2  and  VzN  sessions  using  only  /;1  .  As  before,  session  x  uses  /12  and 
/o  o  .  Every  session  has  heavy  demand,  so  x  has  a  fair  rate  of  l/2  ,  and  the  fair 
rate  for  every  other  session  is  l/N  .  The  window  size  for  each  buffer  of  each 
session  is  at  least  two,  and  2  <  lV(x,  2)  <  14 .V  .  The  round  robin  rings  and 
the  initial  conditions  are  such  that  the  system  has  a  period  of  .V  slots,  j  is 
served  at  /22  during  the  first  half  of  each  period,  and  x  is  served  at  /i  : 
during  the  second  half  of  each  period.  Consequently,  x  transmits  only  U'(j.  2) 
packets  over  each  hop  every  N  slots.  In  other  words,  the  long-term  average 

N 

2-\V{x,  2) 


throughput  of  x  is  unfair  by  a  factor  of 


Moreover,  the  capacity  x 


*  /yv. 


>  V 
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4.4.3  Theorem  3:  Throughput  Bounds  in  Steady  State 


Suppose  a  system  has  been  specified  that  satisfies  the  assumptions  of 
Chapter  2  and  uses  round 'robin  scheduling.  Suppose  each  session  x  has  a 
well-defined,  real  demand  rate  X(x)  ,  0  <  X(x)  <  1  .  Suppose  there  exists  a 
nonnegative  real  number  A  such  that,  for  each  session  x  and  for  all  times  s 
and  t  satisfying  0  <  s  <  t  , 


(201) 


C(x,  0,  s,  t )  —  X(x)*(t  —  s)  |  <  A 


Suppose  that,  for  each  session  x, 

(202)  l)S’Ns~  1*(A+2)  <  W{x,h)  <  oo  for  1  <  h  <  H{x) 

It  follows  that  there  exists  a  time  >  0  such  that,  for  each  session  x,  for 
each  hop  h  of  x  in  the  range  0  <  h  <  H(x)',  and  for  all  times  s  and  t 
satisfying  Tss  <  s  <  t  , 

(203)  C(x,  h,  s,  t )  >  Rc{x,  h)'{t  -  s)  -  {H  +  l)s~  l-Ns~  X-(A  +  2) 


(204)  |  P(x,  A,  s,  0  -  RF(I(x))-(t  -  s)  |  <  [H  +  l)s ’Ns~  *•( A  +  2) 

For  concreteness,  define  T$s  to  be  the  smallest  nonnegative  time  with  this 
property. 


Proof  of  Theorem  3 


In  order  to  prove  (203)  and  (204),  it  will  be  shown  that  there  exist  times 


0  *  Tss(0)  5:  Tss(l)  <  •••  <  Tss(I)  and  such  that  properties  (205)  -  (208) 
hold. 

(205)  For  each  session  x,  for  each  hop  A  of  x  in  the  range  0  <  A  <  i/(x)  , 
and  for  all  times  s  and  t  satisfying  rSJ(/(x)-l)  <  .  <  i  : 

C(z,  k,  s,  t )  >  Rc(z,  *)'((  —  «)  —  fCL(J\,  /( i)) 

(206)  For  each  session  x,  for  each  hop  A  of  x  in  the  range  0  <  A  <  /f(x)  , 
and  for  all  times  s  and  i  satisfying  T55(/(x)-l)<  5<  t: 

F(x,  A,  s ,  0  >  RF(I(x))-(t  -  s)  -  Fpl(A,  I(x)) 

(207)  For  each  session  x,  for  each  hop  A  of  x  in  the  range  0  <  A  <  H(x)  , 
and  for  all  times  s  and  t  satisfying  Tss(I(x)~  1)  <  s  <  t  : 

P{x,  A,  s,  0  <  Ff(/(x))-(«  -  «)  +  F"/>c(A,  I(x)) 

(208)  For  each  session  x,  for  each  hop  A  of  x  in  the  range  0  <  A  <  H(x)  , 
and  for  all  times  s  and  £  satisfying  TssWz))  <‘<t: 

P{ x,  h,  s,  t )  <  Rf(I(x)y(t  -  s)  +  FPU{&,  /(*)) 

The  proof  is  by  induction  on  the  congestion  index  l(x)  of  the  session  x. 
Contrary  to  custom,  the  induction  step  will  be  proved  before  the  base  case  is 
addressed.  Fix  a  congestion  index  i  >  1  .  The  induction  hypothesis  asserts 
that  there  exist  times  Fss(°)  >  rss(l)  ...  ,  Tss(t  —  l)  such  that 
0  “  7ss(°)  <  Tss(l)<  •••  <  755(1  —  1)  and  such  that  properties  (205)  - 


(208)  hold  for  all  sessions  x  with  /(x)  <  i  .  It  must  be  shown  that,  for  such  a 
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time  T$s{i—  1)  >  properties  (205),  (206),  and  (207)  also  hold  for  ail  sessions  x 
with  I(x)  =  i  .  It  must  also  be  shown  that  there  exists  a  time 
Ts§{ 0^  Tss{i~  1)  such  that  property  (208)  holds  for  all  sessions  x  with 
I{x)  =  t  . 

First  consider  (205).  Let  x  be  any  session  with  I(x)  =  i  .  Let  h  be  any 
hop  of  x  in  the  range  0  <  h  <  H[x )  .  Let  s  and  t  be  any  times  satisfying 

Tss{i-1)  <  5  <  t  .  If  hop  h  *  0  ,  it  follows  from  assumption  (201), 

definition  (68),  and  conclusion  (89)  of  Lemma  1  that 

C(x,  0,  s,  t)  >  \(x)‘(t  —  s)  —  A 

-  ftc(x>  °)'(t  ~  s)  -  A 

(209)  >  Rc{ i,  0)-(«  -  s)  -  Fcl(A.  i) 

If  1  <  h  <  H(x )  ,  Lemma  2  will  be  used,  with  K  =  1  ,  sl  —  s  ,  t:  =  t  ,  and 
G  —  Fpu(A,  t  — 1)  .  By  conclusion  (83)  of  Lemma  1,  condition  (98)  of  Lemma  2 
holds.  By  the  induction  hypothesis,  (208)  holds  for  all  sessions  with  congestion 
index  less  than  i.  This  fact,  along  with  conclusion  (87)  of  Lemma  1,  verifies 
condition  (99)  of  Lemma  2.  From  conclusion  (100)  of  Lemma  2  and  conclusion 
(91)  of  Lemma  1,  it  follows  that 

C(x ,  h,  s,  t )  >  Rc(x,  ~  s)  ~  (-v  *“  1)'fpu( »“1)  ~  1 

(210)  >  Rc(x -  A)*(f  -  s)  -  Fci(A,  i) 


V'/'.-'vW'/v'v 

■s.  v  v  .  v 


wV  4* »  '•  *•  N  *  *  “• 


k.% 


This  completes  the  proof  of  (205)  for  the  induction  step. 

Now  (206)  will  be  proved.  Let  x  be  any  session  with  I(x)  —  i  .  Lemma  6 

will  be  used,  with  Tx  —  Tss(i—  1)  ,  T2  =  oo  ,  r  =  Rp(i)  ,  Gx  —  0  ,  and 

G2  —  Fcl(A,  i)  .  To  verify  condition  (160)  of  Lemma  6,  let  h  be  any  hop  of  x 
in  the  range  0  <  h  <  H(x )  ,  let  K  be  any  positive  integer,  and 
let  sx  ,  tx  ,  s2  ,  t2  ,  .  .  .  ,  SR  ,  tK  be  any  times  satisfying 

TF$(i —  1)  ^  si  <  h  <  s2  ^  <  *  *  *  <  S/f  <  .  Recall  that  (205)  was 

just  proved  for  all  sessions  with  congestion  index  t.  It  follows  from  (205)  and 
(70)  that 

C(x,  h,  sk  ,  tk )  >  Rc( x,  h)-(tk  -  sk)  -  Fcl{ A,  0 

(211)  >  RF(i)'(tk  —  st)  —  Fcl{i\ ,  t) 

Summing  (211)  over  k  verifies  condition  (160).  Condition  (161)  of  Lemma  6 
holds  because  of  assumption  (202)  and  definition  (74).  Now  conclusion  (162)  of 
Lemma  6  and  conclusion  (93)  of  Lemma  1  can  be  applied  to  show  that,  for  each 
hop  h  of  x  in  the  range  0  <  h  <  //(x)  and  for  all  times  s  and  t  satisfying 
7>s(i-l)  <  5  <  f  • 

P(x,  h.  s ,  t)  >  Rp(i)’(t  -  s)  -  {{H  +  1)*FCZ>(A.  i)  +  H ] 

(212)  >  RF(i)-(t  -  s)  -  Fpl{ X  0 


This  completes  the  proof  of  (206)  for  the  induction  step. 


.  "  W  •  V  "Ji  ’S’4.  *>  ■!  "J  r\  ’  '. 
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Next  (207)  will  be  proved.  Let  x  be  any  session  with  I(x)  =  i  .  Let  h  be 
any  hop  of  x  in  the  range  0  <  h  <  H(x)  .  Let  s  and  t  be  any  times  satisfying 
Tss(t  —  l)  <  s  <  t  .  Lemma.  7  will  be  used,  with  K  =  1  ,  s1—  s  ,  tx  =  t  , 
Gl  =  0  ,  and  G2  =  Fp^(A,  i)  .  Condition  (167)  of  Lemma  7  follows  from 
assumption  (202).  Condition  (168)  of  Lemma  7  is  obviously  true,  since  K  —  1  . 
Condition  (169)  of  Lemma  7  holds  because  of  assumption  (201).  Condition 

(170)  of  Lemma  7  holds  because  of  conclusion  (81)  of  Lemma  1.  By  the 
induction  hypothesis,  (206)  holds  for  all  sessions  with  congestion  index  less 
than  i.  This  fact,  along  with  conclusion  (85)  of  Lemma  1,  verifies  condition 

(171)  of  Lemma  7  for  those  sessions  y  with  I(y)  <  i  .  For  those  sessions  y 
with  I(y)  =  i  ,  condition  (171)  holds  because  (206)  was  just  proved  for  all 
sessions  with  congestion  index  *.  From  conclusion  (172)  of  Lemma  7  and 
conclusion  (96)  of  Lemma  1,  it  follows  that 

P{x,  h,  s,  t )  <  RF{i)’{t  -  s)  +  {N  -  1)'Fpl(A,  0  +  W'-H  +  A 

(213)  <  RF(i)-(t  -  s)  +  F"PU(A,  0 

This  completes  the  proof  of  (207)  for  the  induction  step. 

Now  (20S)  will  be  proved.  Let  x  be  any  session  with  I(x)  =  i  .  Let  h  be 
any  hop  of  x  in  the  range  0  <  h  <  H{x )  .  Lemma  S  of  Appendix  A.l  will  be 


used,  with: 
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y(u)  =  P(x,  h,  u  —  1,  u) 

G(s,  t)  =  P{x ,  h,  s,  t ) 

T  =  Tss(i-l) 

r  =  RF(i) 


fc 


> 

V- 

V 

V 

V 

? 


/'  -  ^tCA,  o 

/"  -  F"pu( A,  0 
£  *  1 

Condition  (335)  of  Lemma  8  holds  for  all  times  s  and  t  satisfying 
Tss  (i— 1)  <  s  <  t  because  (206)  and  (207)  were  just  proved  for  all  sessions 
with  congestion  index  i.  By  Lemma  8,  then,  there  exists  a  time 
T((x,k)>  Tss(i-1)  such  that,  for  all  times  s  and  t  satisfying 
T({x,  h)<  s<  t  , 

(214)  P(x,  h,  s,  t )  <  Rp(i)’(t  -  s)  +  F PL{ A,  t)  +  1 
Applying  conclusion  (95)  of  Lemma  1  yields: 

(215)  P(x,h,s,t )  <  RF(i)'(t  —  s)  +FPU(A,i) 

Define  T^(i)  as  follows: 

(216)  Tss{i)  =  max  T({x.h ) 

z:  r{z)-  I 

h:  0 <h<Hlz) 

This  proves  (208),  completing  the  induction  step. 

Next  the  base  case  (i.e., 


i  =  1)  will  be  considered.  Note  that 
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TjsO-  1)  —  755(0)  =  0  by  definition.  The  proof  for  the  base  case  is  identical 
to  the  induction  step,  considering  the  following  point.  In  the  induction  step, 
the  induction  hypothesis  was  invoked  to  verify  the  assumptions  of  Lemmas  2 
and  7  for  sessions  y  with  I(y )  <  i  .  For  the  base  case,  there  are  no  such 
sessions  y,  so  verifying  these  assumptions  is  trivial. 

This  completes  the  proofs  of  (205)  -  (208).  From  (205)  -  (208),  definitions 
(74)  -  (76),  and  the  fact  that  /(x)  <  S  for  all  sessions  x,  it  follows  that  (203) 
and  (204)  hold  for  each  session  x,  for  each  hop  h  of  x  in  the  range 
0  <  h  <  H(x)  ,  and  for  all  times  s  and  t  satisfying  TssU)  <s<‘ 

This  completes  the  proof  of  Theorem  3. 


4.4.4  Corollary  2:  Bound  on  Buffer  Level  Range 


It  follows  from  the  assumptions  of  Theorem  3  that,  for  each  session  x, 

(217)  M(x,  h )  -  m(x,  h)  <  2'(H+l)s "NS~  1,(A+2)  for  1  <  h  <  H{x) 

Proof  of  Corollary  2 

Choose  any  times  s  >  Tss  and  t  >  Tss  such  that 

(218)  B(x,  h,  s)  =  m(x,  h ) 

(219)  B[x,  h,  t)  =  M(x,  h) 

Suppose  that  s  <  t  .  (The  proof  for  s  >  t  is  similar  and  will  not  be 
presented.)  Applying  (10)  and  conclusion  (204)  of  Theorem  3  gives  the  desired 
result: 

M(x,  h )  —  m(x,  h)  —  B{x,  h ,  t )  —  B(x,  h,  s ) 

=  P(x,  h—  1,  s,  t )  —  P(x ,  h,  s,  t ) 

<  RF{I{x))'{t  -  s)  +  (H  +  1  )s-Ns~l-(A  +  2) 

-  RF{I{x))-{t  -  s)  -  [H  +  1)5-.V5_,-(A  +  2) 

=  2 -{H  +  1)5-.V5_1-(A  +  2) 


This  completes  the  proof  of  Corollary  2. 


4.4.5  Corollary  3:  Effect  of  Bottleneck  Locations  on  Buffer  Levels 

Suppose  that  the  assumptions  of  Theorem  3  hold.  Let  x  be  some  session. 
Let  J  be  the  number  of  bottleneck  hops  for  x,  and  let  h*x  ,  h*2  ,  ...  »  h*j 
denote  the  hop  numbers  of  the  bottlenecks  of  x,  with 

0  <  <  h*2  <  *  *  *  <  h*j  <  H{x) 

Under  the  assumptions  above,  properties  (220)  and  (221)  hold. 

(220)  For  each  hop  h  of  x  in  the  range  0  <  h  <  H(x)  , 
at  least  one  of  the  following  statements  is  true: 

(a)  h  is  a  bottleneck  hop  for  x 

(b)  m(x,  h)  —  0 

(c)  M(x,h  +  1)  -  W[x,h  + 1) 

(221)  For  each  buffer  h  of  x,  0  <  h  <  if(x)+l  , 

at  least  one  of  the  following  statements  is  true: 

(a)  m(x,  h)  >  0 

(b)  M(x,  h)  <  W(x,h) 

Furthermore,  there  exist  buffers  h'0  ,  h\  ,  ...  ,  h1  j  of  x  (called  crossover 
buffers )  with 

(222) 

0“1  h’0  <  h\  <  h\  <  h\_  <  h'2  < 

<  h*j_ !  <  h'J_l  <  h*j  <  h'j  =  H{x)+ 1 

such  that  properties  (223)  and  (224)  hold. 
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(223)  If  h'j  <  A  <  h*j+l  for  some  j,  0  <  j  <  7—1  ,  then 

(a)  A/(x,  A)  =  !F(x,  A) 

(b)  m(x,  A)  >  0 

(224)  If  h*j  <  A  <  A'y  for  some  j,  1  <  j  <  7  ,  then 

(a)  m(x,  A)  =*  0 

(b)  iV/(x,  A)  <  W(xf  A) 

For  the  crossover  buffers  A';  ,  0  <  j  <  7  ,  no  claim  stronger  than  (221)  is 
made. 

A  combination  of  bottleneck  locations  and  buffer  levels  that  is  consistent 
with  properties  (220)  -  (224)  is  shown  in  Figure  9.  The  figure  shows  the  buffers 
and  hops  of  a  session  x  with  a  16-link  path  and  five  bottleneck  hops.  Each 
buffer  A  is  depicted  as  a  square  whose  shading  gives  information  about 
m(x,  A)  and  A/(x,  A)  .  The  crossover  buffers  A';  are  also  indicated.  Hops 
are  shown  as  lines  between  buffers.  The  heavier  lines  are  the  bottleneck  hops. 
Examine  the  buffers  between  two  successive  bottleneck  hops;  note  that  the 
buffers  upstream  of  the  crossover  buffer  are  sometimes  empty  and  are  never 
full  (for  times  t  >  Tss),  while  the  buffers  downstream  of  the  crossover  buffer 
are  sometimes  full  and  are  never  empty  (for  times  t  >  Tss)- 


FIGURE  9 


Proof  of  Corollary  S 

Property  (220)  will  be  proved  by  contradiction.  Suppose  that  there  is  some 
non-bottleneck  hop  h  of  x  in  the  range  0  <  h  <  H(x )  such  that 

m(x,  h)  >  0 

M(x,  h+ 1)  <  W{x,  h+ 1) 

This  means  that,  for  all  times  t  >  T$s  » 

B(x ,  h,  t)  >  0 
i?(x,  A  +  l,  f)  <  VF(:r,  A  +  l) 

Therefore,  session  x  will  accept  every  chance  offered  to  it  by  the  round  robin 
scheduler  at  hop  h  after  time  Tss  5  for  all  times  t  >  Tss  > 

(225)  P(x,h,Tss,t)  =  C(x,  A,  Tjs.O 

It  follows  from  conclusion  (203)  of  Theorem  3  that,  for  all  times  t  >  Tss  > 

(226)  P(x,  h,  Tss  ,  t)  >  Rc(z,  A)-(i  -  Tss )  -  (H  +  +  2) 

Since  h  is  not  a  bottleneck,  (70)  says  that  Rc{x,  h)  >  ■  Therefore, 

(226)  will  violate  conclusion  (204)  of  Theorem  3  if  t  is  large  enough.  This 
proves  (220). 

Next,  property  (221)  will  be  proved.  For  h  =  0  .  (G02a)  is  true.  For 
h  =  i/(x)+l  ,  (G02b)  is  true.  For  the  remaining  buffers  h  =  1,  2,  ...  ,  H{x )  . 
property  (221)  will  be  proved  by  contradiction.  Suppose  there  is  some  buffer  h 
of  x  in  the  range  1  <  h  <  H(x)  such  that 


v  -y  -;>  -  >  -  /•’>  ■ 
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(227)  m(x,  A )  =  0 

(228)  M(x,  A)  =  W(x,  A) 

It  follows  from  assumption  (202)  of  Theorem  3  that 
M(x,  A)  —  m(x,  A)  =  W(x,  A) 

(229)  >  3 -{H  +  l)s %NS~1-(A  +  2) 

This  contradicts  conclusion  (217)  of  Corollary  2,  proving  (221). 

It  was  given  that  A'0  =  0  and  h'j  =  H(x)+l  .  Let  us  now  define 
crossover  buffer  h'j  for  1  <  j  <  /—I  .  If  there  is  any  buffer  A  in  the  range 
A  * j  <  A  <  A*y+1  such  that  M(x,  A)  <  W^x,  A)  ,  then  let  h'j  equal  the 
largest  such  A  ;  otherwise,  let  h'j  =*  A*;-  +  1  . 

Now,  (223)  will  be  proved.  For  1  <  j  <  /—I  ,  property  (G03a)  follows 
from  the  definition  of  h'j  ,  and  (G03b)  follows  from  (G03a)  and  (221).  For 
j  =  0  ,  i.e.,  for 

(230)  0  -  h'0  <  A  <  h\ 

property  (223)  will  be  proved  by  forward  induction  on  A.  First  consider  the 
base  case,  A  =  1  .  By  (230),  hop  0  is  not  a  bottleneck  hop,  and  since 
m(x.  0)  =  cc  ,  (220)  implies  that 

(231)  M(x,  1)  =  W(x,  1) 


Together,  (231)  and  (221)  imply  that 
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(232)  m(x,  1)  >  0 

This  proves  the  base  case.  The  induction  step  is  also  proved  by  applying  (220) 
followed  by  (221).  This  proves  (223). 

Next,  (224)  will  be  proved  by  backward  induction  on  h.  First  consider  the 
base  case,  viz., 

(233)  h*j  <  h  =  h'j  -  1 

For  j  =  1,  2,  ...  ,  J—  1  ,  the  definition  of  h'j  and  (233)  imply  that 

(234)  M{x,h'j)  <  W{x,h'j) 

Note  that  (234)  also  holds  for  j  *»  J  ,  since  h'j  =  H( x)+l  .  Together,  (220), 
(233)  and  (234)  imply  that 

(235)  m(x,  h'j  —  1)  =  0 
Together,  (221)  and  (235)  imply  that 

(236)  A/(x,  h'j  -  1)  <  W{x,  h'j  -  1) 


This  proves  the  base  case.  The  induction  step  is  also  proved  by  applying  (220) 
followed  by  (221).  This  proves  (224)  and  completes  the  proof  of  Corollary  3. 


4.4.6  Corollary  4:  Existence  of  Pure  Bottlenecks 


It  follows  from  the  assumptions  of  Theorem  3  that  every  session  x  has  at 
least  one  pure  bottleneck  hop  h*  in  the  range  0  <  h*  <  H(x)  . 


Proof  of  Corollary  4 

Let  x  be  some  session.  It  will  be  shown  that  x  has  at  least  one  bottleneck 
hop  h*  in  the  range  0  <  h*  <  H(x )  with  the  following  properties: 

m(x,  h *)  >  0 
M(x,  h*+l)  <  W(x,h*+ 1) 

This  means  that  session  x  will  use  all  its  chances  at  this  hop  h*  after  time 
Tss-  In  other  words,  this  hop  is  a  pure  bottleneck  for  x. 

The  proof  will  be  by  contradiction.  Suppose  that,  for  each  bottleneck  hop 

h*  of  x  in  the  range  0  <  h*  <  H(x)  ,  at  least  one  of  the  following  statements 

is  true: 

m(x,  h*)  =  0 

M[x.  h*+ 1)  =  \V{x,  h*+  1) 

This  assumption,  combined  with  conclusion  (220)  of  Corollary  3.  yields  the 
following  propertv. 


(237)  For  each  hop  A  of  x  in  the  range  0  <  A  <  H{x )  , 
at  least  one  of  the  following  statements  is  true: 

(a)  m(x,  A)  =  0 

(b)  Af(x,  A  + 1)  =  W{x,  A  +  l) 

Now  the  following  claim  will  be  proved. 

(238)  For  each  buffer  A  of  x  in  the  range  0  <  h  <  H(x )  , 

m(x,  A)  >  0 

The  proof  of  (238)  is  by  induction  on  h.  The  base  case  is  true  because 
m(x,0)=*  oo  .  The  induction  step  is  proved  by  applying  (237)  followed  by 
conclusion  (221)  of  Corollary  3. 

For  A  =  H(x )  ,  (238)  and  (237)  imply  that 

(239)  M{x,H{x)+ 1)  =  W{x,H(x)+ 1) 

This  gives  a  contradiction,  since  buffer  H(x)+l  is  never  full. 

This  completes  the  proof  of  Corollary  4. 


4.4.7  Example  4:  Existence  of  Impure  Bottlenecks 

Consider  a  system  that  satisfies  the  assumptions  of  Chapter  2  and  has  the 
layout  shown  in  Figure  10.  The  network  contains  links  fx  x  ,  ,  lo  i  ,  U  2  » 

and  /3  .  (For  each  of  these  links,  there  is  another  link  with  opposite  direction 
that  is  not  shown  in  Figure  10  and  is  used  only  to  return  flow  control  permits.) 
For  j  —  1,  2  ,  there  are  two  sessions  j/;1  and  yj  2  that  use  only  link  l;  x  , 
there  are  two  sessions  yj  3  and  4  that  use  links  lj  x  and  2  ,  and  there  is 
a  session  Xj  that  uses  links  lj  2  and  l3  .  Every  session  in  the  system  has 
heavy  demand;  i.e., 

(240)  C(ij  ,  0,  t- 1,  t)  =  C{yjk  ,  0,  t- 1,  0  =  1 

for  j  —  1,  2,  for  k  =  1,  2,  3,  4,  and  for  all  times  f  >  1  .  The  max-min  fair 
rates  for  ix  and  x2  are  Yz  each,  while  the  other  sessions  deserve  rates  of  :4 
each.  Notice  that,  for  j  =  1,  2  ,  Xj  is  bottlenecked  at  both  its  links.  The 
window  size  for  each  buffer  h  >  1  of  each  session  is  at  least  two  but  finite. 
Table  3  shows  the  buffer  levels  at  time  0.  Round  robin  link  scheduling  is  used. 
For  j  =  1,  2  ,  the  ring  at  lj  x  is  x  ,  y;-  2  ,  3  ,  yj  4  .  The  ring  position  of 

x  at  time  0  is  y  x  4  .  while  the  initial  ring  position  of  l2  x  is  t/22  .  For 
j  =  1.  2  ,  the  ring  at  /;  2  is  j/;- 3  ,  j/;  4  .  j;  .  The  ring  position  of  /12  at 
time  0  is  yl  3  ,  while  the  initial  ring  position  of  /2  2  is  x2  .  The  initial  ring 


position  of  /3  is  xx  . 


This  system  is  eriodic,  with  a  period  of  four  slots.  Table  4  shows  which 


session  uses  each  slot  at  each  link  during  the  interval  (0,  4]  .  While  the 
average  session  throughputs  over  every  system  period  are  max-min  fair,  sessions 
xx  and  x2  each  decline  one  chance  to  use  link  /3  in  every  system  period, 
because  of  a  lack  of  packets.  In  other  words,  /3  is  an  impure  bottleneck  for 
xi  and  x2  .  This  does  not  violate  Corollary  4,  however,  because  xt  and  x2 
have  pure  bottlenecks  /12  and  /2  2  ,  respectively. 


/  *  m  »  • 

,  •.  *.  v  %  s'  s' 

**  -  jT— .  jlT—  » 


’  •  *  •  *  %''  A  *'*  .*•  ,*  . 


r-\v 


-  V  ..-  .VVaS  *.’ 


-  145  - 


Slot  Number 


1 


2 


3 


4 


01.1 


01,2 


01,3 


01, 


01,4 


*1 


01, 


Xo 

4 » 


X  2 


*i 


X] 


x2 


02,3 


02,4 


02,3 


02,4 


02,1 


02, 


TABLE  4.  Link  Users  over  One  Period 


.v.  ■ 


4.4.8  Corollary  5:  A  Property  of  Pure  Bottlenecks 

Given  the  assumptions  of  Theorem  3,  any  link  that  is  a  pure  bottleneck  for 
some  session  must  be  a  pure  bottleneck  for  every  session  bottlenecked  there. 

Proof  of  Corollary  5 

Let  K  be  any  integer  satisfying 

(241)  K  >  2 -(H  +  l)s 'Ns~  !*(A  +  2)  +  2 

The  proof  of  Corollary  5  will  be  by  contradiction.  Let  1  be  a  bottleneck  link 
for  sessions  x  and  y.  Suppose  that  l  is  a  pure  bottleneck  for  x  but  not  for  y. 
In  other  words,  there  is  a  time  after  which  session  x  accepts  every  chance 
offered  to  it  by  the  round  robin  scheduler  at  link  /;  session  y,  on  the  other 
hand,  declines  infinitely  many  chances  to  use  link  l.  Therefore,  there  exist 
times  s  and  t  such  that  Tss  <  s  <  t  and  such  that  x  accepts  every  chance 
at  l  during  (5,  t\  : 

(242)  P'{x,l,s,t )  =  C'[x,  l,  s,  t) 
and  such  that  y  declines  K  chances  at  l  during  (-i.  t]  : 

(243)  P'(y.  1.  s.  t)  =  C\y.  1.  s.  t)  -  K 

By  the  operating  rules  of  the  round  robin  scheduler  at  l,  x  must  receive  almost 
as  many  chances  as  y  during  (s,  t]  ;  specifically: 


(244)  C'(x,  l,  s,  t)  >  C'(y ,  /,  s,  t)  —  1 
Combine  (242),  (244),  (243),  and  (241): 

P'{x,  l,  s,  t)  =  C'(x,  l,  s,  t ) 

>  C'{ y,  l ,  5,  0  -  1 
=  P'{y ,  /,  s,  0  +  /iT-  —  1 

(245)  >  P'(y,  /,  s ,  t)  +  2'(H  +  1)‘9-A'5_1-(A  +  2)  +  1 
Applying  conclusion  (204)  of  Theorem  3  to  (245)  yields: 

(246)  P'{x,  l,  s,  t )  >  RF(I(y))-{t  -  s)  +  {H  +  l)5*iV?-1-(A  +  2)  +  1 

Since  x  and  y  are  bottlenecked  at  the  same  link,  it  follows  from  definition  (64) 
that  I(x)  =  I(y)  .  Substitute  this  into  (246): 

(247)  P'{x,  l,  s,  t )  >  RF{I{x))-{t  -  s)  +  {H  +  1)5-jVs"1-(A  +  2)  +  1 


This  contradicts  conclusion  (204)  of  Theorem  3,  completing  the  proof  of 
Corollary  5. 


4.5  Steady-State  Analysis  of  Bursty  Demand  Case 


This  section  studies  the  long-term  average  session  throughputs  when  the 
sessions  have  independent  Bernoulli  demand  processes.  Suppose  such  a  system 
has  been  fully  specified,  including  its  initial  state.  For  future  reference,  define 


min  W(x,h) 

_  x,h:l<  h<  Ff(x) 

max  W(x,h) 

xfh:l<h<H(x) 


.  For  each  buffer  h  of  each  session  x  such  that 


1  <  h  <  H(x )  ,  suppose  that  the  window  size  W(x,  h )  is  at  least 
12’(H+1)S ’Ns~ 1  but  finite.  Such  a  system  can  be  modeled  as  a  finite  Markov 
chain  [15]  in  which  each  state  represents  one  combination  of  buffer  levels  and 
round  robin  ring  positions.  If  the  demand  rate  X(x)  of  each  session  x  is 
strictly  less  than  one,  then  the  Markov  chain  has  a  single  closed, 
communicating  class  of  states,  f  However,  if  X(x)  ==  1  for  even  one  session  x, 
then  multiple  classes  are  possible.  Even  from  the  given  initial  state,  it  may  be 
possible  to  reach  more  than  one  of  these  classes.  With  probability  one,  the 
system  will  eventually  enter  one  of  these  classes,  after  which,  of  course,  it 


t  Suppose  that  every  session’s  demand  rate  is  strictly  less  than  one.  Number 
the  sessions  Xj  ,  x2  ,  ...  ,  xs  ,  put  the  system  in  an  arbitrary  state,  and 
consider  the  following  sequence  of  events.  First,  every  session’s  demand  is 
zero  for  long  enough  that  every  buffer  h  >  1  of  every  session  empties. 
Then  session  Xj  transmits  a  single  packet  through  the  network,  thereby 
setting  the  round  robin  ring  positions  to  xx  at  each  link  of  its  path.  After 
this  packet  has  left  buffer  //(xj+l  ,  session  x2  transmits  one  packet  over  its 
entire  path,  then  session  x3  has  its  turn,  etc.  Since  the  resulting  state  is 
reachable  from  all  states,  the  Markov  chain  has  a  single  closed, 
communicating  class  of  states. 


cannot  leave  that  class.  For  each  session  x  and  each  closed,  communicating 
class  £  ,  there  exists  a  real  number  r(x ,  £)  with  the  following  property:  given 
that  the  system  eventually  ends  up  in  class  £ ,  the  long-term  average 
throughput  RA(x)  of  session  x  equals  r(x,  £)  with  probability  one.  f  This 
means  that,  given  the  initial  state  v  ,  RA(x)  IS  a  random  variable;  RA( x) 
takes  on  value  r(x,  £)  according  to  the  probability  that  the  system  enters 
class  £  from  state  rj  .  With  probability  one,  RA(x)  —  r(x>  £)  for  some  class  £  . 
Theorem  4  concludes  that,  for  each  session  x  and  each  class  £  ,  r(x,  £)  differs 


from  the  fair  rate  RF(I(x))  by  no  more  than 


74S-(tf+l)25-7V25_1 


cr{  W'f 


In  other 


74  <?•('  HA- 1  N^s~ 1 

words,  i?4(x)  is  within  - * — - . -  of  RF(I(x))  for  each  session 

x,  with  probability  one.  This  means  that  the  session  throughput  rates  can  be 
made  arbitrarily  close  to  the  fair  rates  by  choosing  window  sizes  that  are  of  the 
same  order  of  magnitude  and  are  sufficiently  large.  (Example  5  demonstrates 


t  Let  0  be  some  state  in  some  closed,  communicating  class  £  of  the  Markov 
chain  defined  above,  and  let  0(0)  denote  its  mean  recurrence  time.  (Note 
that  1  <  0(0)  <  oo  ,  since  9  is  recurrent  and  the  Markov  chain  is  finite 
[15].)  By  applying  the  strong  law  of  large  numbers  to  the  recurrence  times  of 
9,  it  is  easy  to  prove  the  following  claim:  given  that  the  system  enters  class 
£  ,  the  long-term  average  number  of  visits  to  state  9  per  unit  time  is  1  / 0(9) 
with  probability  one.  For  each  session  x,  let  0(x,  £)  denote  the  set  of 
states  of  £  in  which  x  has  just  transmitted  a  packet  over  hop  H(x)  (i.e.,  in 
which  buffer  /f(x)-|-L  contains  a  packet),  and  let  r(x,  £)  =  £  l/^(^)  • 

«€©(*. 0 

Given  that  the  system  enters  class  £  ,  the  long-term  average  throughput 
RA(x)  of  session  x  equals  r(x,  £)  with  probability  one. 


why  perfectly  fair  rates  cannot  be  achieved,  in  general,  with  finite  window 
sizes,  f  ) 


The  proof  of  Theorem  4  is  structured  as  follows.  Time  is  divided  into 
intervals  of  fixed  length.  Theorem  2  is  used  to  bound  the  session  throughputs 
during  those  intervals  in  which  the  demands  of  the  sessions  are  fairly  smooth. 
Lemma  9  of  Appendix  A.2  is  used  to  bound  the  frequency  of  such  intervals. 
Together,  these  results  show  that  the  session  throughputs  are  nearly  fair  most 
of  the  time. 


t  Of  course,  infinite  windows  are  not  the  solution:  if  unbounded  queues  build 
up  in  some  buffers,  then  cross-network  delay  is  also  unbounded;  moreover,  a 
session’s  throughput  can  be  (wastefully)  higher  on  hops  upstream  of  such 
buffers  than  on  hops  downstream. 


4.5.1  Theorem  4:  Approximate  Fairness  of  Average  Throughputs 


Suppose  a  system  has  been  specified  that  satisfies  the  assumptions  of 
Chapter  2  and  uses  round  robin  scheduling.  Suppose  that  the  demands  of  the 
sessions  are  independent.  For  each  session  x,  suppose  chances  at  hop  0  form  a 
Bernoulli  process  with  rate  X(x)  ,  0  <  \(x)  <  1  .  Suppose  that,  for  each 
session  x, 


(248) 


12-(H+I)s-Ns~l  <  W{x,h)  <  oo  for  1  <  h  <  H{x) 


Define  a  real  number  at  as  follows: 


min  W(x,  h) 
z,h:l<h<H{z) 


min  W(x,  h) 

Z,h:l<h<H[z) 


Z,h:  1<  A<  H[z) 

It  follows  that,  with  probability  one,  the  long-term  average  throughput  RA(x) 
exists  for  each  session  x  and 


(250) 


„  ,  >  „  w  i  ^  74S’(H  +  1  )2^.iV25-i 

Ra(z)  -  I  <  -  - 


Proof  of  Theorem  4 


Define  a  real  number  A  as  follows. 


(251) 


A  = 


oc-W 


6 -(H  +  1  )S’NS~1 


Since  a  is  at  most  one  and  H,  S  and  N  are  at  least  one,  it  follows  from  (251) 
and  (248)  that 
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(252) 


W'  W' 

W'  +  3  A  -1-  4  <  W  +  —  +  —  <  2  W' 

—  4  6  ~ 


It  also  follows  from  (251),  (249)  and  (248)  that  A  >  2  and  hence 
Zm(H+  l)SmNs~  x*( A+2)  <  S-(H+l)s-Ns~l-A 


=  ct‘W' 


(253) 

Therefore,  for  each  session  x, 


min  W(x,  h) 
s,hi  l<h<H{x) 


(254)  3m(H+l)s 'Ns~  1*(A+2)  <  W(x,  h)  <  oo  for  1  <  h  <  H{x) 


These  inequalities  will  prove  useful  later. 


Now  we  proceed  with  the  proof.  Let  r  be  some  integer  in  the  range 

(255)  <**(  W')1,5  <  t  <  2a-(^')1-5 

Divide  the  time  interval  (0,  oo)  into  non-overlapping  subintervals  of  length  r 
slots:  (0,  r],  (r,  2r],  ....  Label  each  subinterval  "good"  or  "bad"  according  to 
the  smoothness  of  the  sessions’  demands  during  that  subinterval;  a  subinterval 
((&—  l)*r,  &*r]  is  "good  for  session  x  "  if 

(256)  |  C(x,  0,  s,  t )  —  \(xY(t  —  s)  |  <  A 

for  all  times  s  and  t  satisfying  (Ic—  l)*r  <  s  <  t  <  k’r .  A  subinterval  is 
"bad  for  session  x  "  if  it  is  not  good  for  x.  Let  7r(x)  be  the  probability  that  a 
given  subinterval  is  bad  for  session  x.  Lemma  9  of  Appendix  A. 2  can  be 


i  ^  ^  \ v  ™  ™  >~  ™  y* 
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applied,  with  C?(s,  t )  =  C(x,  0,  s,  t)  ,  n  =  X(x)  ,  and  T  =  (k  —  l)*r ,  to 
conclude  that 


(257) 


7r(x)  < 


for  all  sessions  x.  A  subinterval  that  is  good  for  all  sessions  is  simply  called 
"good,"  while  a  subinterval  that  is  bad  for  at  least  one  session  is  called  "bad." 
Let  7r  be  the  probability  that  a  given  subinterval  is  bad.  By  (257), 


*  <  £  *■(*) 

2 


<  £ 
2 


(258) 


St 

A2 


For  every  positive  integer  /C,  let  denote  the  number  of  bad  subintervals 
among  (0,  r],  (r,  2r],  ...  ,  ((K—  l)*r,  K't  ]  .  Since  demands  during  different 
subintervals  are  independent  and  identically  distributed,  the  strong  law  of 
large  numbers  [3]  can  be  applied  to  conclude  that,  with  probability  one, 


(259) 


<1K_ 

K 


7T 


Let  Cl'  be  the  set  of  sample  paths  for  which  (259)  holds.  As  just  mentioned. 


(260) 


PROB  {  H'  }  =  1 


Let  n"  be  the  set  of  sample  paths  for  which  the  long-term  average  throughput 
R^(x)  of  every  session  x  exists.  As  explained  in  the  introduction  to  Section 
4.5, 

(261)  PROB  {  Q"  }  =  1 
It  follows  from  (260)  and  (261)  that 

(262)  PROB  {n'flll"}  =  1 

It  suffices  to  prove  (250)  for  all  sample  paths  in  Q'D  f2"  and  all  sessions  x. 

Let  us  restrict  our  attention  to  a  particular  sample  path  u)  in  fi'D  0"  .  Let 
x  be  any  session.  Let  k  be  any  positive  integer.  The  throughput  of  x  during 
subinterval  ((£—  l)*r,  k'T  ]  will  now  be  analyzed.  Since 

(263)  0  <  P(x,  H(x),  (k—  l)*r,  k-r)  <  r 
and 

(264)  0  <  Rf{I(x))  <  1 

it  follows  that 

(265)  |  P{x,  H(x),  (Jt-l)-r,  k‘r)  -  Rf{I{x))-t  \  <  r 

Bound  (265)  will  be  used  only  for  the  bad  subintervals.  If  {{k—  l)*r.  k'r]  is  a 
good  subinterval,  nicer  throughput  bounds  can  be  obtained  by  using  Theorem 
2,  with  Tj  =  (ifc—  \)'T  and  To  =  hmr+  1  .  Conditions  (ISO)  and  ( 1 S 1 )  of  the 
theorem  are  satisfied  by  (256)  and  (254).  Applying  conclusion  (182)  of  Theorem 
2  and  (252)  yields: 


|  P(x,  H{x),  {k-l )*r,  k-r)  -  Rf(I(x))'t  I  <  {H+  l)s +  3A  +  4) 

(266)  <  2  W'(H+1)S  'N2S~ 1 

Now,  the  throughput  bounds  (265)  and  (266)  for  the  bad  and  good  subintervals 
can  be  added  together  in  the  correct  proportions  in  order  to  bound  the 
throughput  of  x  over  longer  intervals.  For  any  positive  integer  K , 


P(x,  H{x),  0,  K-t)  -  Rf(I(x))'K't 

K't 


|  £  [P(x,  H(x),  {k-l)-T,  k-r)  -PF(/(x))-r]  | 
k-  1 _ 

K’t 

£  |  P{x,  H{x),  (k-l)-T,  k'T)  -  Rf(I(x))'r  I 

k-  1 _ 

K-t 


^  qK-r  +  (K  -  qKy[2W'-{H  +  l^-iV25"1  ] 
-  K‘t 


^  qK-r  +  K-[2W'iH  +  i  j 

“  K’t 


(267) 


qK_  2  \V’-{H  +  l^-jV25'1 
K  +  T 


Since  the  sample  path  <jJ  belongs  to  Cl"  ,  the  long-term  average  throughput 
Ra(x)  exists,  and  since  u;  belongs  to  Cl'  .  (259)  applies.  Therefore,  by 
definition  (11),  (267),  and  (259), 


Ra(x)  -  Rf[I{x))  |  =  |  lim 

t—  00 


lim 

K-+  oo 


-  RF{m) 


Pix ,  H(x),  0,  K‘t) 


-  Rf{I{*)) 


<  lim  i*_  +  lf-N3*'1 

~  K-*oo  K  T 


(268) 


=  7T  + 


2WHH  +  1  )S-N*S-1 


Applying  (258),  (255),  and  (251)  to  (268)  gives  the  desired  result  (250): 

■  o  ,  x  „  xx  ,  ^  5*r  .  2 W’-(H+  l)5*^25"1 
I  &a(x)  ~  rf(I{z))  I  <  ~T  +  - 1 - -T - 

A  T 


<  7 -S‘(H  +  1)2S-I^S~2  2  >(H  +  1  )S-N*S 

-  a.(  w’)  °-5  a*(  W')0-5 


74S-(H  +  D^-iV25"1 


cr(W")C 


This  completes  the  proof  of  Theorem  4. 
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4.5.2  Example  5:  Unfairness  with  Finite  Windows 

Consider  a  system  that  satisfies  the  assumptions  of  Chapter  2  and  has  the 
layout  shown  in  Figure  11.  The  network  contains  links  and  l2  •  (For  each 
of  these  links,  there  is  another  link  with  opposite  direction  that  is  not  shown  in 
Figure  11  and  is  used  only  to  return  flow  control  permits.)  Session  x  uses  /j 
followed  by  U  .  Session  y  uses  only  /j  .  Sessions  zx  and  z2  use  only  U  • 
Sessions  x  and  y  have  heavy  demand;  i.e., 

(269)  C(x,0,  *-l,  t)  -  C{y,0,  t-l,  t)  =  1 

for  all  times  t  >  1  .  (Note  that  these  demand  processes  are  Bernoulli,  with 
rate  one.)  Sessions  zx  and  z2  have  independent  Bernoulli  demand  processes 
with  rate  l4  ;  i.e., 

(270) 

PROB  {  C(z{  ,0,  t-l,  t)  =  l}  =  PROB  {  C{z2 , 0,  t-l,  t)  =  1  }  =  % 

for  all  times  t  >  1  .  The  max-min  fair  rates  for  sessions  x,  y,  zl  ,  and  z2  are 
Vz  ,  Vz  ,  lA  ,  and  '4  ,  respectively.  The  window  size  for  each  buffer  of  each 
session  is  at  least  two.  Round  robin  link  scheduling  is  used. 

Divide  the  time  interval  (0,  oc )  into  subintervals  of  length  12'U’(j.2) 

slots,  viz..  (0.  12*  U'(j,  2)],  (12*  lF(x ,  2)  ,  24*  ll'(x.  2)] .  Since  session  y  will 

accept  every  chance  offered  to  it  by  the  round  robin  scheduler  at  link 


(except  possibly  during  slot  1),  session  x  can  transmit  at  most  6*U'(x,  2) 
packets  over  /1  during  any  of  these  subintervals.  In  other  words',  the  average 


throughput  of  x  over  lx  during  any  of  these  subintervals  is  at  most  Vi  . 

Things  can  be  worse,  however.  Consider  a  particular  subinterval 
(12*(/C —  l)*  W(x,  2)  ,  \2'k'W(x,  2)]  .  Suppose  that 

(271)  C{zl  ,  0,  t-1,  t)  =  C(z2  ,  0,  t-1,  t)  =  1 

for  all  slots  t  in  this  subinterval.  (The  probability  of  this  event  is 
(V4-V4)12  =■  ^y2y8' w(z ■-)  .)  Since  sessions  zl  and  z*  will  accept  every 

chance  offered  to  them  by  the  round  robin  scheduler  at  link  /2  during  the 
subinterval  (except  possibly  during  the  first  slot  of  the  subinterval),  session  x 
can  transmit  no  more  than  4‘W(x,2)  packets  over  /2  during  the  subinterval. 
Window  flow  control  will  permit  session  x  to  transmit  at  most  W[x ,  2)  more 
packets  over  link  lx  than  x  transmits  over  /2  during  the  subinterval;  this 
limits  x  to  a  total  of  5*lk(i,2)  packets  over  lx  .  Therefore,  the  average 
throughput  of  x  over  lr  during  the  subinterval  is  at  most  5/l2  . 

In  summary,  during  a  fraction  (Vi)48'  of  the  subintervals,  the  average 

throughput  of  session  x  over  link  lx  is  at  most  5/l2  ,  and  during  the  other 
subintervals,  the  average  throughput  of  x  over  is  at  most  Vi  .  Therefore. 


-  - 


4.6  Unfairness  with  First-Come-First-Served  Scheduling 

The  examples  in  this  section  demonstrate  that  if  first-come-first-served  link 
scheduling  is  used  instead  of  round  robin  scheduling,  then  max-min  throughput 
fairness  is  not  guaranteed  even  if  the  windows  are  large  and  of  comparable 
magnitude.  In  other  words,  Corollary  1  and  Theorem  4  do  not  hold.  These 
examples  show  that  with  first-come-first-served  scheduling,  the  long-term 
average  throughputs  are  strongly  affected  by  the  relative  window  sizes  of 
competing  sessions  and  by  the  initial  conditions  --  even  if  the  windows  are 
large.  In  Example  6,  the  capacity  of  a  link  shared  by  two  sessions  is  divided 
between  the  sessions  in  proportion  to  their  window  sizes.  Unfair  average 
throughputs  result  if  the  sessions  have  unequal  window  sizes,  no  matter  how 
large  the  windows  are.  Example  7  shows  a  complex  system  with  4:V+4  links 
and  N2+ 1  sessions,  where  N  can  be  any  even  integer  greater  than  ten.  In 
this  example,  the  sessions  have  equal  window  sizes.  Some  initial  conditions 
result  in  fair  average  throughputs.  For  other  initial  conditions,  however,  the 
long-term  average  throughput  of  one  session  is  unfair  by  a  factor  of  more  than 
A'/lO  ,  no  matter  how  large  the  window  size.  It  seems  that  the  problem  of 
selecting  window  sizes  to  achieve  throughput  fairness,  for  general  networks  and 
general  initial  conditions,  is  difficult  and  perhaps  impossible  if  first-come-first- 


served  link  scheduling  is  used. 


4.8.1  Example  8:  Unfairness  with  Unequal  Windows 

Consider  a  system  that  satisfies  the  assumptions  of  Chapter  2.  The 
network  consists  of  two  nodes  joined  by  links  l j  and  / 2  of  opposite  direction. 
Link  1 1  is  used  only  by  sessions  x  and  y.  Both  sessions  have  heavy  demand; 
i.e., 

(273)  C(x,0,  t-1,  t)  =  C{y,0,t-l,t)  =  1 

for  all  times  t  >  1  .  Obviously,  the  max-min  fair  rate  for  each  session  is  Y2  . 
Suppose  that  the  window  sizes  W(x,  1)  and  lF(x,  2)  for  session  x  equal  wx  , 
the  window  sizes  W{y,  1)  and  W(y,  2)  for  session  y  equal  wy  ,  and 
wx  =£  wy  .  The  initial  buffer  levels  are  as  follows: 

B{x,  0,  0)  =  oc  B{x,  1,0)  =  wx  -  1  B{x,  2,  0)  =  1 

B{y,  0,0)  =  00  5(y,l,0)  =  wy  B(y,  2,0)  =  0 

First-come-first-served  link  scheduling  is  used.  The  tie-breaking  list  for  link 
is  arbitrary.  The  transmitter  queue  for  ll  initially  contains  wx—  1  reservations 
for  x  and  wy  reservations  for  y,  in  arbitrary  order. 

The  evolution  of  this  system  is  very  simple.  During  slot  1.  session  1 
transmits  a  packet  over  hop  2  (i.e.,  the  packet  is  retrieved  by  the  session's 
sink),  the  corresponding  permit  for  buffer  2  is  returned  upstream  to  hop  1,  and 
a  new  reservation  for  x  is  added  to  the  tail  of  the  transmitter  queue  at  link  / 1  . 
Similarly,  during  the  slot  immediately  following  the  transmission  of  a  packet 


over  /j  ,  that  packet  is  transmitted  over  hop  2,  its  permit  for  buffer  2  is 
returned  upstream  to  hop  1,  and  a  new  reservation  for  that  packet’s  session  is 
added  to  l^s  transmitter  queue.  Hence  link  /j  operates  periodically,  with 


period  wx  +  wy  .  In  each  period,  sessions  x  and  y  transmit  wz  and  wy 
packets,  respectively,  over  .  Therefore,  the  long-term  average  session 
throughputs  are  wx/(wx  +  wy)  and  wy/(wx+wy)  ,  respectively.  Since 
wx  #  wy  ,  these  average  throughputs  are  unfair,  f 


t  It  may  seem  that  the  conclusion  of  this  example  depends  on  the  somewhat 
arbitrary  way  that  session  sources  and  sinks  were  modeled  in  Chapter  2. 
However,  it  is  easy  to  embed  this  example  in  one  with  longer  session  paths,  so 
that  the  interesting  features  occur  at  intermediate  hops. 
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4.6.2  Example  7:  Unfairness  with  Equal  Windows 

Let  N  be  an  even  integer  greater  than  ten.  Consider  a  system  that  satisfies 
the  assumptions  of  Chapter  2  and  has  the  layout  shown  in  Figure  12.  For 
i  —  1,2  and  j  =  1,  2,  ...  ,  VzN,  the  network  contains  links  /l  0  ,  /(  j  l  ,  and 
/,•  j  2  .  (For  each  of  these  links,  there  is  another  link  with  opposite  direction 
that  is  not  shown  in  Figure  12  and  is  used  only  to  return  flow  control  permits.) 
For  t  =  1,  2  and  j  =  1,  2,  ...  ,  VzN,  there  is  a  session  y,  j  x  that  uses  links 

!  ,  l{ j  o  ,  and  li  0  ,  and  there  are  N—l  sessions  *  Vi,j, 3  >  •••  >  Vi.j.N 

that  use  only  1,  j  x  .  There  is  also  a  session  x  that  uses  /10  followed  by  L  0  . 
Every  session  in  the  system  has  heavy  demand;  i.e., 

(274)  C{x,  0,  t-1,  t )  =  C(y{J  k  ,  0,  t- 1,  t)  -  1 

for  i  =  1,  2,  for  j  =  1,  2,  ...  ,  VzN,  for  k  =  1,  2,  ...  ,  N,  and  for  all  times 

t  >  1  .  The  max-min  fair  rate  for  session  x  is  l/2  ,  while  the  other  sessions 

deserve  rates  of  l/N  each.  The  windows  for  ail  buffers  h  >  1  of  all  sessions 
have  the  same  size  w  >  2  .  Table  5  shows  the  buffer  levels  at  time  0.  First- 
come-first-served  link  scheduling  is  used.  The  tie-breaking  lists  are  arbitrary. 
At  time  0,  the  transmitter  queue  for  link  ll  Q  contains  exactly  one  reservation 
each  for  sessions  V\j\,  J  —  1,  2,  ...  ,  V2.V.  and  possibly  some  reservations  for 
session  x.  The  transmitter  queue  for  0  may  initially  contain  any  number  of 
reservations  for  sessions  x  and  y2,j,  1  »  j  =  1,  2,  ...  ,  V2N,  as  long  as  all 


reservations  for  sessions  (/2  x 


are  in  the  first  2 w  queue  positions.  Let  us 


X 

oo 

vo  or 

w—  1 

between 
0  and  w 

y  i,j,i 

1  <  j  <  VzN 

oo 

w  —  1 

1 

VlJ.k 

1  <  j  <  VzN 

2  <  k  <  N 

oo 

w 

0 

l<  j  <  VzN 

oo 

w 

0 

V2J,k 

1  <  j  <  VzN 

2  <  k  <  N 
k  #  VzN  +  1 

oo 

w 

0 

y2,j,%N+\ 

1  <  j  <  VzN 

oo 

w—  1 

1 

%N 

Note:  £]  B(y2  j  i,  3,  0)  <  2 w 

1 


TABLE  5.  Initial  Buffer  Levels 


-  168  - 


now  describe  the  initial  transmitter  queues  at  links  j  t  2  ,  U  ;  \  ,  and 
/o  j  2  >  f°r  i  =  1*  2,  ...  ,  %iV  .  The  transmitter  queue  for  llj  j  contains  w  —  2 
reservations  for  J/ij',1  i  followed  by  w  reservations  for  yl 2  *  then  w; 
reservations  for  y1  j  3  ,  ...  ,  then  ty  reservations  for  yx ^  ,  and  finally  one 
reservation  for  y\j\*  The  transmitter  queue  for  /j 2  contains  one 
reservation  for  y1;1.  The  transmitter  queue  for  /2  j  l  contains  ty  — 2 
reservations  for  y2  iAjv+1  ,  followed  by  ty  reservations  for  y2  2  »  —  >  then 
ty  reservations  for  y*  j,N  >  then  ty  reservations  for  y2  y  i  »  •••  >  then  ty 
reservations  for  >  and  finally  one  reservation  for  y2 ;  !4iV+ 1  •  The 

transmitter  queue  for  I2;2  is  initially  empty,  f 

This  system  will  be  analyzed  over  successive  time  intervals  of  length  iV-iy 
slots.  It  will  be  shown  that  the  throughput  of  session  x  during  [l,  /V’ty]  is  less 
than  5ty  packets.  It  will  also  be  shown  that  the  buffer  levels  and  transmitter 
queues  at  time  N'w  satisfy  the  same  assumptions  that  were  made  for  their 
initial  values,  so  that  x’s  throughput  bound  during  [l,  ;V*ty]  also  applies 
during  [iV-tu+l  ,  2 iV*ty],  (2^V,iy+l  ,  3iV-ty],  ....  Table  6  shows  which  session 

t  In  practice,  this  "initial"  system  state  could  arise  if  sessions  ?/,  ;  <  started  at 
various  times  before  time  0,  when  there  were  already  many  other  sessions 
using  links  /1;1  and  /2;1  ,  and  if  these  extra  sessions  terminated  before 
time  0. 


uses  each  slot  at  each  link  during  [1,  N'w]  ,  for  the  case  where  N  *=  12 
and  w  =«  3  . 

Let  us  examine  the  operation  of  links  l1Q  and  /20  during  the  first  half  of 
the  study  interval  [l,  N‘w)  .  At  time  0,  the  transmitter  queue  for  li  0 
contains  at  most  w  packets  for  session  x  and  exactly  one  packet  for  each 
session  Vij  \  ,  /  =  1,  2,  ...  ,  VzN  .  During  [l,  w—  l]  ,  fewer  than  w  packets 
for  x  and  exactly  w—  1  packets  for  each  session  y1;>1  are  added  to  this 
queue.  In  other  words,  the  total  number  of  packets  to  enter  this  queue  by  time 
w—  1  is  at  least  hkN’w  but  fewer  than  (%Ar+2)*'u;  .  These  packets  will  be 
transmitted  over  /10  before  any  later  arrivals  for  session  x  are  served. 
Consequently,  the  packets  for  sessions  y1;-  r  are  guaranteed  to  be  transmitted 
over  /10  by  time  ( VzN+2)’w  ,  well  before  the  end  N-w  of  the  study 

interval.  Furthermore,  the  throughput  for  x  over  /10  during  [l,  VzN’w]  is 

limited  to  the  (fewer  than  2 w)  packets  joining  lx  0’s  transmitter  queue  by  time 
w—  1  .  Therefore,  the  buffer  capacity  constraint  will  restrict  x's  transmission 
over  lo  o  during  [l,  VzN'w]  to  fewer  than  2w  +  w  =  3w  packets. 

A  similar  analysis  is  possible  for  the  second  half  of  the  study  interval 
[l,  A'* m]  .  It  will  be  shown  that  intense  competition  from  other  sessions 
impedes  session  x's  flow  on  its  second  link,  while  the  window  mechanism 
impedes  the  flow  on  its  first  link.  During  [VzN’w—  1  ,  (V2.V+  —  2]  ,  link 

lo  j  !  transmits  w  packets  for  session  y2,;,i  >  J  =  1>  2,  ...  ,  V2 N  .  The  initial 

conditions  guarantee  that  by  time  2 in  link  i20  finishes  transmitting  any 


>  V  'A 


packets  for  session  that  were  waiting  initially.  Therefore,  a  full  supply 

of  w  permits  for  buffer  3  is  available  at  hop  2  when  the  w  new  packets  arrive 
over  lo  j  i  ,  and  these  packets  immediately  join  /2  ;-  2’s  transmitter  queue. 
During  [ VzN'w  ,  (V^^-h  l)*w —  l]  ,  these  packets  are  transmitted  over  /2;2 
and  join  the  transmitter  queue  at  /20  .  At  time  VzN'w  ,  there  at  most  w 
packets  for  session  x  in  /20’s  transmitter  queue.  During 
[teN'w  +  l  ,  (V2N+l)’w—  l]  ,  fewer  than  w  packets  for  x  are  added  to  the 
queue.  These  packets  for  sessions  y2;1  and  x  ~  totalling  at  least  VzN'w 
packets  but  fewer  than  (V2N+2)’w  —  will  be  transmitted  over  /20  starting  at 
slot  bkN'w+l  ,  and  they  will  all  be  transmitted  before  any  later  arrivals  for 
session  x  are  served.  Consequently,  the  packets  for  sessions  Vi.j.i  are 
guaranteed  to  be  transmitted  over  f20  by  time  (iV+2)*u;  .  (In  other  words, 
any  reservations  for  sessions  y2 j  t  in  12  0  s  transmitter  queue  at  the  end  N'w 
of  the  study  interval  must  be  in  the  first  2w  queue  positions.)  Furthermore, 
the  throughput  for  x  over  /20  during  [ VzN'w+l  ,  N’w ]  is  limited  to  the 
(fewer  than  2 w)  packets  that  were  present  in  /20’s  transmitter  queue  at  time 
VzN'w  or  joined  it  during  [VLV’uz+I  ,  (!/2.V+  1)* w—  l]  .  Therefore,  the  buffer 
capacity,  constraint  restricts  x's  transmission  over  /10  during 
[V2.\r,w-r  1  ,  iY*ir]  to  fewer  than  3  «.•  packets. 


This  completes  the  analysis  of  links  ll0  and  /2  0  .  The  operation  of  links 
/,•  j  j  and  lj j  2  i  *  =  1 ,  2,  j  =  1,  2,  ...  ,  V2N,  during  [l,  N'w\  is  simpler. 
Link  /,•  j  !  transmits  a  block  of  w  packets  for  each  session  y,  ;  k  in  turn. 


During  the  slot  immediately  following  a  packet’s  transmission  over  ;  j  ,  that 
packet  is  transmitted  over  hop  2,  its  permit  for  buffer  2  is  returned  upstream 
to  hop  1,  and  a  new  reservation  is  added  to  j ^’s  transmitter  queue.  (For 
session  >  hop  2  is  link  y  2  •  For  each  session  j /,  j  *  ,  &  #  1,  hop  2  is 

the  session’s  sink.)  Recall  that  j’s  packets  are  transmitted  over  /,•  0  in 
plenty  of  time  to  get  their  permits  for  buffer  3  back  to  hop  2  before  the  next 
batch  of  packets  for  y,-  ;  t  arrive  over  /,•  y  :  .  Hence  the  transmitter  queues  for 
links  /,■  ,  !  and  2  are  periodic,  with  period  N'w  . 

In  summary,  over  the  entire  study  interval  [l,  iV*w]  each  session  Vij  k  > 
i  =  1,  2,  j  —  1,  2,  ...  ,  y2iV,  k  —  1,2,  ...  ,  iV,  transmits  exactly  w  packets 
over  link  /,  j  i  ,  while  session  x  transmits  fewer  than  2w  +  3w  —  ow  packets 
over  each  of  its  links.  Since  the  system  satisfies  the  initial  condition 
assumptions  again  at  time  N’w  ,  these  throughput  claims  also  hold  for  time 
intervals  [N’w+l  ,  2N’w],  [2N‘w+ 1  ,  3iV*ti>],  ....  Therefore,  the  long-term 
average  throughputs  of  sessions  y,- ;  k  are  max-min  fair,  but  the  long-term 
average  throughput  of  i  is  less  than  5/N  ,  which  is  significantly  lower  than  its 
fair  rate  of  1/2  .  j  In  other  words,  Ms  average  throughput  is  unfair  by  a  factor 
of  more  than  X/ 10  ,  regardless  of  the  window  size  tr .  Moreover,  the  capacity 

A  The  long-term  average  session  throughputs  must  exist  because  this  system 
has  a  finite  number  of  states  and  is  deterministic.  Eventually  the  system  wiil 
enter  some  state  it  has  already  visited,  after  which  the  system  will  be 
periodic. 
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lost  by  x  (viz.,  more  than  l/2  —  5 JN  at  each  of  its  links)  is  not  used  by  the 
other  sessions  ~  it  is  wasted,  f 

The  unfairness  in  this  example  depends  critically  on  the  unfortunate  initial 
conditions.  Other  initial  conditions  for  this  same  network  can  result  in  fair 
average  throughputs.  For  example,  suppose  that  the  transmitter  queue  for  link 
Z10  at  time  0  contains  w  reservations  for  session  i  alternating  with  tv—  1 
reservations  for  session  ■  The  transmitter  queue  for  l2Q  initially 

contains  one  reservation  for  yo^,w  i  •  The  initial  transmitter  queues  for  the 
other  links  are  arranged  so  that  packets  for  session  Vi  j  \  ,  *  =  1,2, 

j  =  1,  2,  ...  ,  VzN  ,  are  transmitted  over  link  2  during  slots 

(i+2wj— 2w),  (i+2wj—2w+2),  (t  +  2wj—2w+4),  ...  ,  (i+2wj—  2)  .  In 
other  words,  packets  competing  with  session  x  arrive  in  smooth  streams,  during 
the  odd-numbered  time  slots  at  link  l\  o  and  during  the  even-numbered  slots 
at  U  0  .  Therefore,  during  the  interval  [l,  iV*u/]  ,  session  x  transmits  one 
packet  across  /10  during  each  odd-numbered  slot  and  one  packet  across  L  o 
during  each  even-numbered  slot.  By  correctly  setting  the  initial  conditions,  the 
entire  system  can  be  made  periodic  with  period  N’w  ,  so  that  these  smooth 
flows  continue  forever  and  the  long-term  average  throughputs  are  fair. 

4  It  may  seem  that  the  conclusion  of  this  example  depends  on  the  somewhat 
arbitrary  way  that  session  sources  and  sinks  were  modeled  in  Chapter  2. 
However,  it  is  easy  to  embed  this  example  in  one  with  longer  session  paths,  so 
that  the  interesting  features  occur  at  intermediate  hops. 


5.  SESSION  THROUGHPUTS  IN  SYSTEMS  WITH  SMALL  WINDOWS 


This  chapter  studies  the  throughput  of  a  particular  session  x  in  a  system 
with  bounded-delay  link  scheduling.  The  window  sizes  for  session  x  are 
assumed  to  be  at  least  two.  Except  for  Vf(x,  0)  and  possibly  W(x,  l)  ,  the 
windows  for  session  x  are  assumed  to  be  finite.  The  window  sizes  of  the  other 
sessions  in  the  network  are  arbitrary.  Session  x  is  assumed  to  have  a  well- 
defined,  real  demand  rate  X(i)  in  the  range  0  <  \(x)  <  1  .  The  detailed 
demand  assumptions  for  x  vary  from  section  to  section.  The  demands  of  the 
other  sessions  are  arbitrary,  except  for  the  possible  restriction  that  these 
demands  be  independent  of  the  demand  of  session  x.  These  other  sessions 
need  not  have  well-defined  demand  rates.  Clearly,  the  assumptions  of  this 
chapter  are  much  less  restrictive  than  those  of  Chapter  4.  The  results  of  this 
chapter,  therefore,  are  of  greater  practical  value. 


Theorem  5  assumes  that  Vb'(x,  I)  is  at  least  two  but  finite.  The  demand 
of  session  x  is  modeled  as  a  Bernoulli  process  that  is  independent  of  the  other 
sessions'  demands.  The  theorem  concludes  that  the  throughput 
P{ x,  H(x).  0.  t)  of  x  is  bounded  below  by  a  function  <t>(M  whose  long-term 


average  rate  equals  i  with  probability  one)  7 


\  i  1 ) 


1  —  V x  \  '  +  A  lx >> 


As  one 


\,  x 


■’ o u i d  expect,  this  guarani-"- 1  rate  ’ends  to  z<*ro  as  the  demand  rate  \p 


tends  to  zero  or  as  the  schedule  delay  bound  .4(x)  tends  to  infinity.  .As  \(x) 
tends  to  one,  the  guaranteed  rate  tends  to  l/.l(x)  ,  and  as  .4(x)  tends  to 


one,  the  guaranteed  rate  tends  to  X(x)  ;  these  limits  are  also  intuitive. 


Theorem  6  shows  that  the  guaranteed  rate  can  be  increased  by  allowing 
session  x  to  buffer  more  of  its  demand:  this  theorem  assumes  W(x,  1)  to  be 
infinite.  The  demand  assumptions  are  a  little  weaker  than  before:  the  times 
between  chances  for  session  x  at  hop  0  are  only  assumed  to  be  independent 
and  identically  distributed,  and  the  demand  of  session  x  may  be  dependent  on 
the  demands  of  the  other  sessions.  Theorem  6  concludes  that  the  throughput 
P(x,  H(x),  0,  t )  of  x  is  bounded  below  by  a  function  whose  long-term 

average  rate  equals  (with  probability  one)  min[l/.4(x)  ,  X(x)]  .  This  is 
obviously  the  largest  rate  guarantee  possible  if  nothing  else  is  known  about  the 
scheduling  discipline;  a  better  guarantee  cannot  be  achieved  in  general  by 
assuming  window  sizes  larger  than  two. 


It  was  explained  in  Section  2.4.3  that  round  robin  scheduling  and  first- 
come-first-served  scheduling  are  bounded  delay  disciplines,  with  schedule  delay 
bounds  A'(l )  of  Nr(l)  and  N'(l)’W"—  \V"+ 1,  respectively.  Therefore,  the 
rate  guarantees  of  Theorem  5  for  round  robin  systems  and 


first-come-first-served  systems  are 


X(x) 


X(x) 


, ...  i 


l  —  A  I  X 


■  'Slz'-  W-  »'  +  l  ,  r 


+  Lv(x)-ir"-  ir"+i;-x(j) 


[l  —  X(x  )p  1  ‘  +  -Y(x  )*X(x) 

,  respectively.  The  rate 


guarantees  of  Theorem  6  are  min[l/.Y  '(x)  .  X(x)]  ami 

min’ l/f  .Y(x)-  U’"—  \V"+ 1)  ,  X(i)]  ,  respectively.  Example  1  of  Section  3.2 
■a  -  i  round  robin  system  with  a  session  x  whose  demand  rate  X(x)  =  1  and 
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whose  long-term  average  throughput  is  l/Ar(x)  .  Example  2  of  Section  3.3 
shows  a  first-come-first-served  system  with  a  session  x  whose  demand  rate 
X(z)  —  1  and  whose  long-term  average  throughput  is  l/[N(x)'\V"—\V"+l\  . 
The  average  throughputs  in  these  examples  match  the  bounds  of  Theorems  5 
and  6.  Note  that  the  throughput  guarantees  of  Theorems  5  and  6  for  round 
robin  systems  are  superior  to  those  for  first-come-first-served  systems.  (It  is 
not  being  claimed  that  round  robin  scheduling  always  offers  larger  throughputs 
or  fairer  throughputs  than  first-come-first-served  scheduling.) 

Let  us  compare  the  throughput  guarantees  of  Theorem  6  to  the  max-min 
fair  rates  defined  in  Section  4.1.  Consider  a  system  that  satisfies  the 
assumptions  of  Theorem  6  and  has  a  demand  process  so  regular  that  the  long¬ 
term  average  throughputs  /^(z)  ,  the  demand  rates  X(z)  .  and  hence  the 
max-min  fair  rates  RF(I(x)}  exist  for  all  sessions  x.  Obviously,  for  each 
session  z.  R.\{x)  <  X(z)  <  1  and  R/riJix'))  <  Xizj  <  1  .  Real!  fr  m; 
Section  4.1  that  every  session  x  has  at  least  one  bottleneck  h<  p.  By  i'-fnl'l  n> 
(64)  and  (65).  this  means  that  either  Ry,  [ix  =  x  :r  i  ha.-  a  1  ‘  •  t. 

link  /  'U'*h  t  hat 
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rf(i(x))  =  i  -  e  Rp(i(y)) 

y  using  / 
y  #  x 


>  i  -  2  (/(*)) 

y  using  l 
y  +  x 


-  1  -  [AT'(0  -  l]-fif(/(x)) 


(275) 


>  1  -  [JV(i)  -  !]•«,(/(*)) 


and  hence  RF(I(x))  >  l/.Y(x)  .  If  the  system  uses  round  robin  link 
scheduling,  Theorem  6  guarantees  that  Ra(x)  >  min[l/iV(x  )  ,  X(*)]  . 
Combining  these  various  results  shows  that  if  X(x)  <  l/N(x)  ,  then 
Rf(I(x))  X(x)  —  /^(x)  ,  and  if  X(x)  >  l/.Y(x)  ,  then 
l/.Y(x)  <  RF(I(x))  <  1  and  l/.V(x)  <  /^(x)  <  1  and  hence 
1 

<  — _ — . -  <  .\(x)  .  In  either  case,  the  long-term  average  throughput 


•Yf  x )  Ra(x) 

of  x  is  within  a  factor  of  .Yi'x)  of  its  fair  rate.  The  analogous  guarantee  for 

•  i  scheduling  involves  a  factor  of  ,\(x  )*  U’"—  U'"-f  1  . 


.  iru-ser'. 
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s .-heduling.  Example  7  ..f  Section  1.6. ‘J  shows  an  unfairness  factor  proportional 


to  N(x)  and  Example  2  of  Section  3.3  shows  an  unfairness  factor  roughly 
equal  to  W"  .  f 


For  ease  of  exposition,  it  was  assumed  in  Examples  3  and  7  that  U'ix.1;  i; 
finite.  However,  the  long-term  average  session  throughputs  for  thes» 
examples  are  the  same  whether  U'(j,  1)  is  finite  or  infinite. 


^  VVVST  V".  V.. 


r  *.4 
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5.1  Theorem  5:  Throughput  Bound,  given  Finite  Demand  Buffer 

Suppose  a  system  has  been  specified  that  satisfies  the  assumptions  of 
Chapter  2  and  uses  a  bounded  delay  scheduling  discipline.  Let  x  be  some 
session.  Suppose  that 


(276) 


2  <  W(x,  h)  <  oc  for  1  <  h  <  H(x)+ 1 


The  window  sizes  of  the  other  sessions  in  the  network  are  arbitrary  (i.e.,  these 
window  sizes  only  need  to  satisfy  the  basic  assumptions  of  Chapter  2). 
Suppose  that  chances  for  session  x  at  hop  0  form  a  Bernoulli  process  with  rate 
X(x)  ,  and  0  <  X(x)  <  1  .  Suppose  that  the  demand  of  session  x  is 
independent  of  the  demands  of  the  other  sessions  in  the  network;  except  for 
this  restriction,  the  demands  of  the  other  sessions  are  arbitrary.  It  follows  that 
there  exist  random  variables  $(1),  $(2),  ...  such  that,  for  all  times  t  >  1  . 


(277) 


P(x,H(x),  0,  t)  >  <l>(0 


and  such  that,  with  probability  one, 


(-7S) 


y  _ Xf  i ) _ 

t  —  *7  '1  —  \i  jV  *  +  A  i  r  i-\(  x' 


In  other  w-'.t  is.  til'1  av-r:»g“  throughput  l’  i .  //  x  .  t  >/:  •  f  ^--h  u  x  : 


mage  art  :■  ru 


~  *  1  r '  t  V  ,’l  f  i 


'l  —  X|  i  )j  ’  +  .1  ■  x  jv\i  x 


>'-v  gnmi-T 


taking  t  sufficiently  large.  (This  cannot  b"  phrased  in  terms  of  the  long-term 
average  throughput  of  x;  i.e.,  we  cannot  say: 


*  V.  •„ 


wy^FmJfwjrmJwXmJT^  V  V  V.v.w/^YJ7.  y.  v. 
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Ra(x)  —  l*im 

*— ►  OO 


[x,  (x),  0, 


[1  —  X(x)]'4^^  +  A(x)*X(x) 


since  the  limit  above  may  not  exist.) 


Proof  of  Theorem  5 


First  let  us  clarify  the  scheduling  assumptions.  It  follows  from  (21)  and 
(276)  that,  for  all  packets  p  >  1  of  x  and  all  hops  h  in  the  range 
1  <h<  H(x)  , 

T(x,  k,  p) 

<  max  [  T(x,  h  —  l,  p),  T(x,  h,  p  —  l),  T(x,  A  +  l,  p—  W(x,  h  +  l))  ]  +  A{x.  h 

<  max  [  T(x,  A  — 1,  p),  T(x,  h,  p  —  l),  T(x,  A  +  l,  p—  W(x,  A  +  l))  ]  +  A{x) 
(279) 

<  max  [  T(x,  h  —  l,  p),  T(x,  h,  p  —  l),  T(x,  A+l,  p  — 2)  ]  +  A{x) 


Next  the  demand  assumptions  will  be  clarified.  Let  p'  be  the  first  packet  of 
session  x  to  be  transmitted  over  hop  0  after  time  0.  For  all  packets  p  >  1  . 
define  rp  as  follows. 


(280) 


=  T(x.  0.  p  i  —  max  [  T(x.  0.  p  —  l  x  Tt  x.  1 .  p  —  U  x.  1 


.Note  that  rp  —  0  for  1  <  p  <"  p'  .  For  p  >  p'  .  r 


*  *1'*  1  :i ’•  *  r  Hi  t  H 


time  the  network  is  prepared  to  accept  packet  p  (i.e.,  when  all  packets  older 
than  p  have  been  transmitted  over  hop  0  and  there  is  room  in  buffer  1  i  until 


packet  p  is  actually  transmitted  over  hop  0.  Since  the  demand  of  session  x  is 
Bernoulli  with  rate  X(x)  and  is  independent  of  the  other  sessions’  demands,  it 


follows  that  Tpi  ,  Tpi+l  ,  ...  are  independent  and  identically  distributed 
according  to  a  geometric  distribution  with  mean  l/X(x)  : 

(281)  PROB  {  Tp  =  k  }  =  X(x)*[l— X(x)]*- 1  for  p  >  p',  k  >  1 


For  each  hop  h  of  x  in  the  range  0  <  h  <  H(x)  and  every  integer  p, 
define  ©(A,  p)  as  follows. 


(282)  0(A,p) 


E  max[r?  ,  ^4 (x)]  +  h-A{x) 
?- 1 

0 


for  p  >  1 
for  p  <  0 


Note  that 


(283)  ©(A  — 1,  p)  +  -4(x)  =  ©(A,  p)  for  1  <  h  <  H[i),  p  >  1 

and 

(284)  ©(0,  p  —  1)  +  rp  <  ©(0,  p)  for  p  >  1 
and 

(28.')  ©(A-l.p)  >  Slh.p-l)  for  1  <  h  <  Hix'.  p  >  0 

It  follows  from  (281)  that 

(2Sf>) 

EXPECT  A  TIO.X  {  mux’%  ..lx'}  *  J  ---•**  r  ;•  j 

MX) 

It  follows  from  (282).  the  strong  law  of  larg<*  n  umbers  3  .  an  i  2sr>  'hit.  w  ;•  a 


probability  one,  lim  0(H(x),p)/p  exists  and 

p  — *  00 


lim 

p—  00  P 


E  maxK  .  <*(*)] 


=  lim 


p  — ♦  oo 


E  max[r?  ,  A(x)] 


=  lim 

p— ♦  OO 


E  max[r?  ,  A(x)J 
lim  - - - 

(p-p'+ 1)— 00  P  —  P  +  1 


(287) 


—  Ax 


x  *A  x, 


Let  us  prove  the  following  claim: 


(2S8)  T(x,  A,  p)  <  G(k,  p)  for  0  <  A  <  H(x),  p  >  -1 


The  proof  is  by  induction  on  p.  The  base  cases  p  =  —  1  and  p 


trivial: 


T  x .  h  .  —  1  =  i  =  l  i  i\ :  r  ( )  <  h  <  // !  x  i 


T  x .  /;.<»  =  H  = 


!'  r  0  <  h  <  // 1  x 


. *  '  I  •  ‘  1  {  1  “ 


assert  that 


(291)  T(x,  h,  p  —  2)  <  0(A,  p  — 2)  for  0  <  h  <  #(x) 

and 

(292)  T(x,  h,  p  —  1)  <  &(h,  p  —  1)  for  0  <  h  <  /f(x) 

It  must  be  shown  that 

(293)  •  T(x,  h,  p)  <  ©(A,,  p)  for  0  <  h  <  ff(x) 

The  proof  of  the  induction  step  will  itself  be  an  induction,  this  time  over  h. 
For  the  base  case  (i.e.,  h  —  0),  First  apply  (280)  and  (276): 

T(x,  0,  p)  =  max  [  T(x,  0,  p-1),  T(x,  1,  p—  W(x,  1))  ]  +  rp 

(294)  <  max  [  T(x,  0,  p-1),  T(x,  1,  p-2)  ]  +  r? 

Now  apply  (292),  (291),  (285),  and  (284)  to  (294)  to  reach  the  desired 
conclusion: 

T(x,  0,  p)  <  max  [  ©(0,  p-l),  ©(1,  p-2)  j  +  rp 

=  ©(0,  p-1)  +  rp 

(295)  <  ©(0,  p  ) 

For  the  induction  step,  consider  a  hop  A  of  i  in  the  range  1  <  h  <  H[x)  . 
(The  case  h  =  H(i)  will  be  treated  separately.)  The  induction  hypothesis 
asserts  that 

(296)  T(x,  h- 1,  p)  <  9{h-  1,  p) 


It  must  be  shown  that 


(297)  T (x,  A,p)  <  ©(A,p) 

From  (279),  induction  hypothesis  (296)  (for  the  induction  on  A ),  and  induction 
hypotheses  (292)  and  (291)  (for  the  induction  on  p),  it  follows  that 

(298) 

T(x,  A ,  p)  <  max  [  T(x,  A  — 1,  p ),  T(x,  A,  p  —  1),  T(x,  A  +  l,  p  —  2)  ]  +  .4  (x) 

(299)  <  max  [  ©(A  — 1,  p),  ©(A,p  —  1),  ©(A  +  l,  p  — 2)  ]  +  A(x) 

Applying  (2S5)  and  (283)  to  (299)  gives  the  desired  result  (297): 

T(x,  h,  p)  <  ©(A  —  l,  p)  +  A(x) 

-  e(A\p) 

The  proof  for  the  remaining  case,  viz.,  A  =  H(x)  ,  is  similar,  but  inequality 
(12)  must  be  used  to  handle  the  term  T(x,  A  +  l,  p  —  2)  in  (298)  above.  The 
proof  of  this  case  will  not  be  presented.  This  completes  the  proof  of  (293)  by 
induction  on  A,  thereby  completing  the  proof  of  (288)  by  induction  on  p. 

For  all  times  t  >  1  ,  define  $(<)  as  follows: 

(300)  d>(<)  =  max  {  p:  Q(H[x).  p)  <  t  } 

Note  that  (277)  follows  from  (300),  (2SS)  and  the  fact  that 

(301)  P(x,  H(x),  0,  t)  =  max  {  p:  T(x,  H(x),  p)  <  t  } 

Also  note  that,  for  all  times  t  >  1  , 


(302) 


e(H{x),  $(t)+l)  >  t  >  0(ff(x),  <J>(f)) 


and  hence 


(303) 


wo ± n - 1  <  m.  < 

e(H(i),  *(0+0  -  t  -  .  e(H(x),  *(()) 


Since  $(£)  ^  oo  and  Q(H(x),  p)  ^  oo  ,  it  follows  from  (303)  and 

t— *  oo  p—  00 


(287)  that,  with  probability  one,  lim  $(£)/£  exists  and 

f  — *  00 


Bm  _ 1 - 

‘  Hm  e! 

p—oo  p 

«  _ _ 

[l  —  X(x)]"4^  +  A(x)*X(x) 


This  is  the  desired  result  (278),  completing  the  proof  of  Theorem  5. 
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5.2  Theorem  6:  Throughput  Bound,  given  Infinite  Demand  Buffer 

Suppose  a  system  has  been  specified  that  satisfies  the  assumptions  of 
Chapter  2  and  uses  a  bounded  delay  scheduling  discipline.  Let  x  be  some 
session.  Suppose  that  buffer  1  of  x  has  infinite  capacity: 

(304)  W(x,  1)  =  oc 
but  that 

(305)  2  <  W(x,  k)  <  oo  for  2  <  h  <  H(x)+ 1 

The  window  sizes  of  the  other  sessions  in  the  network  are  arbitrary  (i.e..  these 
window  sizes  only  need  to  satisfy  the  basic  assumptions  of  Chapter  2). 
Suppose  that  the  times  between  chances  for  session  x  at  hop  0  are  independent 
and  identically  distributed,  with  mean  l/X(x)  ,  and  0  <  X(x)  <  1  .  (Because 
of  (304),  each  such  chance  will  result  in  a  packet  transmission.)  The  demands 
of  the  other  sessions  in  the  network  are  arbitrary.  It  follows  that  there  exist 
random  variables  $(1),  $(2),  ...  such  that,  for  all  times  t  >  1  , 

(306)  P{x,  H(x),  0,  t )  >  *{t) 
and  such  that,  with  probability  one, 

— t —  .  \  i  j  i 
.I'.C 

In  other  words,  the  average  throughput  P(x.  //(x),  0,  t)/t  of  session  i  can  be 
made  arbitrarily  close  to  min  l/.llrl  .  X(x)j  --  or  greater  --  by  taking  t 


307 


4>m 


min 


'/•-  P .*■  .*•  A 


* .  ■  .  »  _  *  •  • 

^ ^ e . jT.  ..v 


■Cj.VVO. 


sufficiently  large.  (This  cannot  be  phrased  in  terms  of  the  long-term  average 
throughput  of  x;  i.e.,  we  cannot  say: 


n  <  \  ,•  P(x,  H{x),  0,  t)  ^  .  1  w  , 

Ra[x)  =  hm  — A ^ - L  >  mm  .  X(x) 

t  — ►  oo  t  .4  ( x  j 

since  the  limit  a^ove  may  not  exist.) 

Proof  of  Theorem  6 

First  let  us  clarify  the  scheduling  assumptions.  It  follows  from  (21)  and 
(305)  that,  for  all  packets  p  >  1  of  x  and  all  hops  h  in  the  range 
1  <  h  <  H(x)  , 

T(x,  h,  p ) 

<  max  (  T(x,  h—  1,  p),  T(x,  h,  p  —  1),  T(x,  A  +  l,  p—  lF(x.  A  +  l))  )  +  .4^x.  h 

<  max  [  T(x,  h—  1,  p),  T(x,  h,  p  —  1),  T(x,  A  +  l,  p—  VV(x,  h  + 1))  J  -f  .4(x) 

(308) 

<  max  [  T(x,  h—  1,  p),  T(x,  h,  p  —  1),  T(x,  A  +  l,  p  —  2)  ]  +  .4(x) 

Next  the  demand  assumptions  will  be  clarified.  Let  p'  be  the  first  packet  of 
session  x  to  be  transmitted  over  hop  0  after  time  0;  i.e., 

(309)  T(x,  0,  p)  =  0  for  1  <  p  <  p' 

(310)  T(x,  0,  p)  >  0  for  p  >  p' 

For  p  >  p'+l  ,  let  r  be  the  delav  between  the  transmission  times  at  hop  0 


7?/; 


kWltSL'd 
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of  packets  p—  1  and  p  for  session  x 


r  =  T(x.  0.  p )  —  T(x,  0,  p  —  1)  for  p  >  p'+l 


It  is  given  that  T(x.  0,  p'),  ty+ 1  ,  rp/+  2  ,  ...  are  independent  and  that  r,  +  j  . 
~p+2  ’  •••  are  identically  distributed,  with  mean  l/X(x)  .  (Note  that  the  time 
T(x.  0.  p ')  until  the  first  transmission  over  hop  0  may  have  a  different 
distribution  and  a  different  mean  than  the  intertransmission  times  7-.+  1< 


~p’+2  ,  ••••  )  Also  define 


r  =  A(x)  for  1  <  p  <  p' 


It  follows  from  (311)  and  (312)  that,  for  all  packets  p  >  1  , 


(313) 


T(x,  °,  p)  <  T(z,  0.  ?')  +  7-j  +  r2  + 


For  each  hop  h  of  x  in  the  range  0  <  h  <  H(x )  and  every  integer  p 
define  ©(A,  p)  as  follows. 


(31-4) 


0  ( h  ,  p  )  = 


N._l-  that 


T(x.  0.  p')  +  (p  +  A)\4(x)  4-  max 

i<y<5 


0i  h  —  I .  :> 


f“+  *  —  ]  "> 


i  l.l  i,l  l 


r,** 


It  follows  from  definitions  (31 4)  and  (312)  that,  for  all  p  >  p' , 


Q(H(x),  p)  —  T(x,  0,  p')  +  (p  +  tf(x)]*A(x)  +  max  £  [r,  -A(z)] 

1</<P  y—  p'+ 1 


T(x,  0,  p')  +  [p  +  tf(x)]*A(x)  +  max  £  lr;  --*(*) 

p'<^<p  y-p'+i 


(317) 


=  T(x,  0,  p')  +  [p  +  tf(x)]\4(x)  +  max  £  [r  ,+  jfc  -A(x)] 

0 <K<p-p'  kmml 


From  (317)  and  Lemma  10  of  Appendix  A.3  it  follows  that,  with  probability 


one,  lim  0(ff(x),  p)/p  exists  and 

p—  OO 


« 


=  A(x)  +  lim 

V  fo-n'W 


o<  “<?-,■  *5,  [v+*  ~ A(z)] 


(?-p')— 00 


P  -  P 


A(  x)  +  lim  max 

{p-p')—*oo  o<  K<  p—p' 


E  [y+4  -A(x)} 

k-  1 _ 

P  -  p' 


=  A(x)  +  max 


A(x) 


(318) 


—  max 


A(x) 


1 

X(x) 


Let  us  prove  the  following  claim: 


(319) 

T(s,  h,  p)  < 

©(A,  p) 

for 

VI 

o 

VI 

H(x), 

P  >  - 1 

The  proof 

is  by  induction 

on  p. 

The  base  cases 

P  -  ~ 

■  1  and 

trivial: 

(320) 

T(x,  A,  — 1)  = 

■■  0  = 

©(A .  — 

1) 

for 

VI 

o 

<  #(*) 

(321) 

T(x,  ft,  0)  = 

0  = 

©(A,  0) 

for 

-c; 

VI 

o 

s 

h: 

VI 

For  the  induction  step,  consider  a  packet  p  >  1  .  The  induction  hypotheses 


assert  that 
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(322)  T(x,  A,  p- 2)  <  ©(A,  p- 2)  for  0  <  A  <  H{x) 

and 

(323)  T(x,  A,  p  —  1)  <  ©(A,  p  —  1)  for  0  <  A  <  H{x) 

It  must  be  shown  that 

(324)  T(x,  A,  p)  <  ©(A,  p)  for  0  <  A  <  H{x) 

The  proof  of  the  induction  step  will  itself  be  an  induction,  this  time  over  A. 
The  base  case  (i.e.,  A  *  0)  follows  from  (313)  and  definition  (314): 

T(z,  0,  p)  <  r(x,  0,  p')  +  rx  +  r2  +  •  •  •  +  rp 

=  T(x,  0,  p’)  +  p*A(z)  +  £  [tj  -A(x)] 

j-i 

J 

<  T(x,  0 ,  p’)  +  pvt(x)  +  max  £  [r.-  —  A(x)] 

I<'<?  1 

(325)  *  0(0,  p) 

For  the  induction  step,  consider  a  hop  A  of  x  in  the  range  1  <  A  <  H(x)  . 
(The  case  A  =  H(x)  will  be  treated  separately.)  The  induction  hypothesis 
asserts  that 

(326)  T(x,  A  — 1,  p)  <  ©(A  —  1,  p) 

It  must  be  shown  that 


.'  A  /.  V 


From  (308),  induction  hypothesis  (326)  (for  the  induction  on  A),  and  induction 
hypotheses  (323)  and  (322)  (for  the  induction  on  p),  it  follows  that 

(328) 

T (x,  A,p)  <  max  [  T(x,  h  —  1,  p),  T(x,  A,  p  —  1),  T(x,  A  +  l,  p  —  2)  ]  +  A{x) 

(329)  <  max  [  ©(A  —  1,  p),  ©(£,p  — l),  ©(A  +  l,  p  —  2)  ]  +  A(x) 

Applying  (316)  and  (315)  to  (329)  gives  the  desired  result  (327): 

T (x,  A,p)  <  ©(A  — 1,  p)  +  A(z) 

-  ©(A,  p) 

The  proof  for  the  remaining  case,  viz.,  K  *=  H(x )  ,  is  similar,  but  inequality 
(12)  must  be  used  to  handle  the  term  T(x,  A  +  l,  p  —  2)  in  (328)  above.  The 
proof  of  this  case  will  not  be  presented.  This  completes  the  proof  of  (324)  by 
induction  on  h,  thereby  completing  the  proof  of  (319)  by  induction  on  p. 

For  all  times  t  >  1  ,  define  $(f)  as  follows: 

(330)  $(t)  =  max  {  p:  p)  <  t  } 

Note  that  (306)  follows  from  (330),  (319)  and  the  fact  that 

(331)  P(x,  H(x),  0,  t)  =  max  {  p:  T(x.  H(x),  p)  <  t  } 

Also  note  that,  for  all  times  t  >  1  , 

(332)  ©(tf(x),  <*>(0+1)  >  1  >  e(//(x),  <^(0) 


and  hence 


(333) 


l«(0  +1I-1  <  ML  <  WO 

e(H(x),  WO+0  -  f  ~  e(ff(*),*(0) 


Since  <f>(£)  '  oo  and  Q(H(x),  p )  '  oo  ,  it  follows  from  (333)  and 

<—►00  p— *00 


(318)  that,  with  probability  one,  lim  $(£)/f  exists  and 

<—►00 


lim  ^ 

<— ►  oo  £ 


lim  gigfekEl 

P—  oo  p 


max 


A(l)  ’  X(*) 


min 


A(x) 


,  X(i) 


This  is  the  desired  result  (307),  completing  the  proof  of  Theorem  6. 


6.  CONCLUSIONS 


Round  robin  scheduling  with  windows  is  a  simple  technique  for  allocating 
link  capacity  among  competing  sessions  in  a  packet  network.  If  a  sufficiently 
large  window  size  is  used  throughout  the  network,  the  session  throughput  rates 
can  be  made  arbitrarily  close  to  the  ideal  max-min  fair  rates.  (A  session 
requiring  a  very  large  throughput  rate  could  be  visited  more  than  once  in  each 
polling  cycle,  effectively  treating  it  like  several  standard  sessions.)  This 
performance  is  suited  to  file  transfers  and  other  applications  where  throughput 
rate  is  of  greater  concern  than  packet  delay.  For  a  session  using  small 
windows,  however,  the  round  robin  method  guarantees  a  small  cross-network 
delay  for  each  packet  while  still  guaranteeing  a  certain  minimum  service 
rate,  f  This  service  rate  determines  the  maximum  session  throughput  rate  that 
can  be  supported  and  also  roughly  determines,  for  a  given  throughput  rate,  the 
delay  of  packets  waiting  to  be  admitted  to  the  network.  (A  session  requiring  a 
larger  guaranteed  service  rate  and/or  a  smaller  guaranteed  cross-network  delay 
could  be  visited  more  than  once  in  each  polling  cycle.)  $  This  performance  is 


t  In  order  for  these  guarantees  to  be  meaningful,  the  routing  algorithm  should 
limit  the  number  of  sessions  sharing  any  link  and  the  number  of  hops  in  any 
session’s  path. 

X  While  reducing  a  session’s  window  size  should  decrease  the  cross-network 
delay  of  that  session’s  packets,  it  might  also  increase  the  time  the  packets 
spend  waiting  to  enter  the  network.  Thus  the  total  delay  (i.e.,  pre-network 
plus  cross-network)  could  actually  increase  as  the  window  size  decreases. 
(Mukherji  [21]  shows  this  for  a  different  flow  control  scheme.)  Small 
windows  do  offer  certain  advantages  to  the  session,  however.  The  session 
becomes  aware  of  congestion  problems  earlier  and  can  respond  to  large  pre¬ 
network  queues  by  compressing  its  data,  prioritizing  its  data  (and  deferring 
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suited  to  interactive  data,  packet  voice,  and  other  applications  requiring  low 
packet  delays.  These  guarantees  for  sessions  with  small  windows  apply  even  if 
other  sessions  in  the  network  are  using  larger  windows.  Hence  this  scheme 
should  be  well  suited  to  integrated  services  networks,  i.e.,  those  supporting  a 
variety  of  applications.  Delay-sensitive  sessions  can  use  small  windows  to  meet 
their  needs,  and  the  remaining  transmission  capacity  can  be  fairly  divided 
among  the  other  sessions  by  assigning  them  large  windows. 

It  should  be  mentioned  that  the  performance  of  the  round  robin  method 
improves  as  the  packet  size  (used  by  all  sessions)  is  decreased.  If  the  window 
sizes  —  measured  in  packets  —  are  fixed,  then  the  cross-network  delay 
(measured  in  seconds  per  packet)  drops  as  the  packet  size  is  reduced.  If  the 
window  sizes  —  measured  in  bits  —  are  fixed,  then  the  throughput  fairness 
improves  as  the  packet  size  is  reduced.  Of  course,  these  beneficial  effects  are 
balanced  by  the  fact  that  packet  overhead  is  more  significant  for  small  packet 
sizes. 

This  thesis  shows  that  round  robin  scheduling  with  windows  compares 
favorably  to  first-come-first-served  scheduling  with  windows,  f  This  finding  is 

or  discarding  the  low  priority  items),  or  requesting  a  higher  service  rate  from 
the  network.  Moreover,  the  component  of  total  delay  that  cannot  be  directly 
observed  or  controlled  by  the  session,  viz.,  the  cross-network  component,  is 
guaranteed  to  be  small  if  small  windows  are  used. 

t  In  modeling  first-come-first-served  scheduling,  this  thesis  assumes  that  link- 
by-link  windows  are  used  and  that  a  packet  may  not  join  a  link  transmitter 
queue  until  it  obtains  a  window  permit  for  its  next  buffer.  Perhaps  a 


of  practical  interest,  since  window  flow  control  is  commonly  used.  The  max- 
min  fairness  results  for  round  robin  scheduling  with  large  windows  do  not 
apply  for  first-come-first-served  scheduling.  Even  when  large  windows  are 
used,  the  session  throughput  rates  in  a  first-come-first-served  system  depend 
strongly  on  the  relative  window  sizes  of  competing  sessions  and  the  initial 
conditions  of  the  '  ATien  small  windows  are  used,  the  throughput  and 

delay  guarantee  for  round  robin  systems  are  also  better  than  those  for  first- 
come-first-served  systems.  (This  is  not  to  say  that  round  robin  scheduling 
performs  better  in  every  case.  I  have  seen  systems  where  first-come-first-served 
scheduling  produces  slightly  fairer  throughputs.)  Moreover,  these  first-come- 
first-served  guarantees  for  a  session  x  depend  on  the  window  sizes  of  the  other 
sessions,  whereas  the  round  robin  guarantees  depend  only  on  x’s  window  size. 
Hence  round  robin  scheduling  simplifies  the  problem  of  selecting  window  sizes. 

A  simplistic  description  of  the  capacity  allocation  mechanisms  of  these  two 
disciplines  may  help  explain  why  their  throughput  performance  is  so  different. 
First-come-first-served  scheduling  allocates  link  capacity  to  sessions  according 
to  the  average,  number  of  packets  each  session  has  waiting.  Round  robin 
scheduling,  on  the  other  hand,  considers  the  fraction  of  time  each  session  has 
at  least  one  packet  waiting. 

different  implementation  of  first-come-first-served  scheduling  would  perform 

better. 


This  thesis  assumed  that  the  propagation  delays  of  the  network  links  were 
negligible.  The  difference  in  worst-case  throughput  performance  for  round- 
robin  and  first-come-first-served  scheduling  should  be  even  more  pronounced  if 
propagation  delays  are  significant.  Consider  a  system  that  uses  first-come- 
first-served  scheduling  and  has  the  layout  shown  in  Figure  13.  Session  x  uses 
links  lx  and  Z2  .  Sessions  yx  ,  y2  ,  ...  ,  y^-i  also  use  Z2  •  Flow  control 
permits  for  the  sessions  are  returned  over  links  /3  and  Z4  ,  whose  directions 
are  opposite  to  lx  and  Z2  ,  respectively.  Suppose  that  the  round  trip 
propagation  delay  over  links  lx  and  Z3  is  d  times  the  length  of  a  packet 
transmission  slot,  while  the  propagation  delay  over  the  other  links  is  negligible. 
Suppose  that  the  window  size  for  every  session  except  x  is  w  packets.  If 
sessions  x,  yx  ,  ...  ,  yk~l  have  very  high  demand  and  sessions  yk  ,  ...  ,  y^-i 
have  very  low  demand,  and  if  each  active  session  is  to  receive  its  fair 

throughput  of  packets  per  slot,  then  W(x,2)  should  be  roughly  ~T  +  w 

fC  K 

packets.  The  fair  window  size  for  one  value  of  k  is  unfair  for  other  values,  and 
the  inequity  worsens  as  the  propagation  delay  grows.  I  conjecture  that  this 
problem  is  much  less  severe  if  round  robin  scheduling  is  used  instead  of  first- 
come-first-served  scheduling. 

A  similar  difficulty  arises  if  end-to-end  windows  are  used  instead  of  link- 
by-link  windows.  Consider  a  system  that  uses  first-come-first-served 
scheduling  with  end-to-end  windows  and  has  the  layout  shown  in  Figure  14. 
Session  x  uses  links  Zj  ,  /2  and  /3  and  shares  each  link  l,  with  single-hop 


IS 


sessions  yt  l  ,  yi2  ,  •••  ,  Vi,N-  \  •  (For  each  of  these  links,  there  is  another  link 
with  opposite  direction  that  is  not  shown  in  Figure  14  and  is  used  only  to 
return  flow  control  permits.  The  propagation  delays  of  all  links  are  negligible.) 
Suppose  that  N  is  very  large  and  that  the  window  size  for  every  session  except 
x  is  w  packets.  If  all  sessions  have  heavy  demand,  then  session  x’s  window  size 
should  be  approximately  3w  packets  in  order  to  achieve  its  fair  throughput 
rate  of  l/N  packets  per  slot.  If  sessions  y11  ,  ...  ,  j  ,  y2  j  ,  ...  ,  y2jV-i 
have  very  low  demand,  however,  and  sessions  x,  y3  j  ,  ...  ,  y^N-i  have  high 
demand,  then  x’s  window  size  should  only  be  about  w  packets.  Window  sizes 
that  are  fair  for  one  scenario  are  unfair  for  the  other,  and  the  inequity  worsens 
as  the  path  length  grows.  I  conjecture  that  this  problem  with  end-to-end 
windows  is  much  less  severe  if  round  robin  scheduling  is  used  instead  of  first- 
come-first-served  scheduling. 

For  a  final  comparison  of  these  two  disciplines,  consider  a  system  with  link- 
by-link  windows.  Suppose  one  session  adjusts  its  window  size  to  optimize  its 
own  throughput  and  delay.  Suppose  that  this  session  has  a  great  many 
competitors  with  large  windows.  I  conjecture  that  the  network  appears  very 
different  to  such  a  session  when  round  robin  scheduling  is  used  than  when 
first-come-first-served  scheduling  is  used.  I  suspect  that  with  round  robin 
scheduling  the  session  can  vary  its  delay  over  a  very  wide  range  and  can  vary 
its  throughput  between  zero  and  the  max-min  fair  rate.  With  first-come-first- 
served  scheduling,  however,  I  suspect  that  the  session  can  vary  its  throughput 
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over  a  very  wide  range,  even  taking  unfair  fractions  of  link  capacity,  but  can 
hardly  control  its  delay  at  all.  The  round  robin  scenario  seems  preferable  to 
me.  These  first-come-first-served  problems  are  compounded  if  all  the  sessions 
are  adjusting  their  window  sizes.  A  session  primarily  interested  in  maximizing 
its  throughput  will  increase  its  window  size.  If  its  competitors  follow  suit,  no 
one  will  get  any  more  throughput  and  everyone’s  cross-network  delay  will 
increase.  Eventually  the  delay  increases  may  halt  this  game,  but  I  believe  that 
round  robin  scheduling  would  produce  an  earlier  truce. 

The  costs  and  benefits  of  the  round  robin  strategy  can  also  be  compared 
with  the  other  fair  flow  control  schemes  mentioned  in  Section  1.1.  Bially, 
Gold,  and  Seneff  [2],  Jaffe  [16,  17],  Gerla  and  Staskauskas  [11:  Section  3], 
Hayden  [13:  Chapters  4-5],  Gafni  [5:  Chapter  3],  Gafni  and  Bertsekas  [6], 
Oshinsky  [22],  and  Mosely  [20]  propose  algorithms  and  session  source  controls 
to  achieve  max-min  fair  rates.  Since  these  schemes  actually  compute  the  max- 
min  fair  rates,  they  can  accomodate  more  variations  in  the  objective  function 
than  are  possible  with  the  round  robin  strategy.  For  one  thing,  link  delays  can 
be  considered  in  the  optimization.  The  round  robin  method’s  provisions  for 
delay  ‘management,  viz.,  window  size  adjustment  and  polling  frequency 
adjustment,  are  less  systematic  but  are  more  suitable  when  different  sessions 
have  different  delay  requirements.  To  compare  the  costs  of  round  robin  and 
the  other  approaches,  the  computation  and  communication  overhead  of  the 
max-min  algorithms  plus  the  difficulty  of  source  rate  control  (i.e.,  variable  rate 
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vocoding  or  variable  packet  sizing  or  packet  metering)  must  be  weighed  against 
the  accounting  burden  of  managing  round  robin  schedules  and  windows  plus 
the  cost  of  the  link  capacity  needed  to  transmit  window  permits.  The  round 
robin  method  compares  more  favorably  if  session  demands  change  frequently  or 
if  session  lifetimes  are  short,  since  the  max-min  algorithms  must  be  executed 
often  under  those  circumstances. 

The  round  robin  strategy  should  also  be  compared  with  the  approaches  of 
Golestaani  and  Gallager  [12,  8],  Gerla  and  Staskauskas  [ll:  Section  5.2], 
Thaker  and  Cain  [26],  Ibe  [14],  Gafni  [5:  Chapters  4-6],  and  Sauve,  Wong  and 
Field  [24,  25].  The  round  robin  method  has  less  overhead,  because  neither  the 
target  session  rates  nor  the  window  sizes  nor  the  schedule  parameters  need  to 
be  computed,  but  round  robin  is  limited  to  a  smaller  variety  of  throughput 
objectives.  While  Golestaani  and  Gallager,  Gerla  and  Staskauskas,  and  Thaker 
and  Cain  also  include  in  their  objectives  the  cross-network  delay  averaged  over 
all  sessions,  they  cannot  easily  accomodate  individual  sessions  with  stringent 
delay  requirements.  The  round  robin  scheme  is  better  at  this.  Sauve,  Wong 
and  Field  (in  a  related  paper  [27]),  Ibe,  and  Gafni  solve  this  problem  by  using 
various  forms  of  priority  queuing.  The  overhead  costs  of  window  flow  control 
apply  to  all  the  strategies  of  this  paragraph.  A  round  robin  schedule  itself  may 
be  slightly  harder  to  execute  than  the  first-come-first-served  schedules 
apparently  assumed  by  Golestaani  and  Gallager,  Gerla  and  Staskauskas,  and 
Thaker  and  Cain,  but  it  is  no  more  difficult  than  the  priority  scheduling  of  Ibe 


and  Gafni,  and  it  is  much  easier  than  the  schedules  of  Sauve,  Wong  and  Field, 
which  depend  on  each  packet’s  real-time  delay. 

Mukherji’s  flow  control  strategy  [21]  is  extremely  flexible.  By  correctly 
setting  periodic  link  schedules,  virtually  any  feasible  set  of  session  throughput 
rates  can  be  enforced,  with  very  small  cross-network  delays.  The  difficulty,  of 
course,  is  in  determining  the  desired  set  of  throughputs.  Optimizing  the 
fairness  of  the  target  throughputs  or  minimizing  the  delay  built  into  the  link 
schedules  could  incur  overhead  penalties  the  round  robin  method  does  not 
have.  However,  with  almost  no  effort  Mukherji  can  offer  throughput  and  delay 
guarantees  comparable  to  the  small-window  guarantees  of  the  round  robin 
method.  In  fact,  since  Mukherji  recommends  round  robin  re-scheduling  of 
transmission  slots  not  claimed  by  their  rightful  owners,  the  performance  of  the 
two  strategies  should  be  similar  in  many  applications.  The  overhead  for 
executing  the  schedules  and  enforcing  the  windows  in  the  two  schemes  should 
be  comparable  as  well.  Mukherji’s  method  also  has  the  property  (described 
earlier  for  round  robin)  that  an  individual  session  can  choose  its  window  size  to 
suit  its  cross-network  delay  requirements. 

This  thesis,  in  its  examples  as  well  as  its  analysis,  focused  on  the  worst-case 
performance  of  round  robin  scheduling  with  windows.  An  important  area  for 
future  study  is  the  typical  performance  of  the  scheme.  Of  interest  are  the 
following  items,  as  functions  of  the  window  size:  the  fairness  of  the  session 
throughput  rates,  the  burstiness  of  the  session  flows,  the  severity  of  transients 


arising  from  the  initiation  and  termination  of  sessions  and  from  changes  in 
session  demands,  and  the  cross-network  delay.  Unfortunately,  since  many  of 
these  performance  measures  are  very  sensitive  to  the  network  topology,  the 
session  routes  and  demand  rates,  and  the  initial  conditions,  such  a  study  would 
likely  require  the  simulation  of  a  great  many  sample  networks  of  at  least 
moderate  size.  It  would  also  be  interesting  to  see  whether  the  use  of  end-to- 
end  windows  instead  of  link-by-link  windows  significantly  changes  the 
performance  of  the  strategy.  Link  models  with  propagation  delays  and  unequal 
capacities  could  also  be  considered.  Finally,  it  would  be  worthwhile  to  examine 
variations  of  this  method  to  see  if  max-min  fair  throughput  rates  can  be 
achieved  without  computing  the  rates  but  without  incurring  large  cross- 
network  delays.  One  approach  is  to  dynamically  adjust  the  window  sizes  so 
that  they  are  no  larger  than  necessary.  Another  possibility  is  to  change  the 
round  robin  discipline  slightly,  e.g.,  by  randomly  rearranging  the  polling  order 
of  the  sessions  from  time  to  time.  This  might  ensure  that  the  system  enters 
very  unfair  configurations  only  rarely  and  only  for  brief  periods. 


A.l  Lemma  8:  Symmetry  of  Upper  and  Lower  Bounds  in  Steady  State 

Let  g[')  be  a  real  function  of  an  integer  argument.  Define  a  real  function 
G(v)  with  integer  arguments  as  follows: 

(334)  G(s,  0  -  b  *(«) 

U—  4+  1 

(Note  that  G(s,  t)  —  0  for  s  >  t  .)  Let  T  be  some  integer,  and  let  r,  f, 
and  f"  be  some  real  numbers.  Suppose  that,  for  all  integers  s  and  t  satisfying 
T  <  s  <  t  , 

(335)  -  /'  <  G(s ,  t)  -  r*(f  -  s)  <  f" 

It  follows  that,  for  every  positive  real  number  e  ,  there  exists  an  integer 
Tf>T  such  that,  for  all  integers  s  and  t  satisfying  T{  <  s  <  t  , 

(336)  —  min  [/',  (/"  +  e)j  <  G(s,  t)  —  r*(t  —  s)  <  min  ^/w,  (/'  +  e)j 

Proof  of  Lemma  8 
Assume  that 

(337)  f<f" 

(The  proof  for  f  >  f"  is  similar  and  will  not  be  presented.)  Because  of 
assumptions  (337)  and  (335),  most  of  (336)  is  trivially  true.  All  that  must  be 
proved  is  this:  for  every  positive  real  number  e  ,  there  exists  an  integer 
T(>  T  such  that,  for  all  integers  s  and  t  satisfying  T(  <  s  <  t  , 


The  proof  will  be  by  contradiction.  Suppose  that  there  exists  a  positive  real 
number  e  such  that,  for  all  integers  T  >  T  ,  there  exist  integers  s  and  t  such 
that  T  <  s  <  t  and  such  that 

(339)  G(s,  t)  >  r-{t  -  s)  +  /'  +  6 

For  any  positive  integer  K ,  this  supposition  can  be  applied  repeatedly  to 
construct  a  sequence  of  integers  Sj  ,  tx  ,  $2  ,  t2  >  •••  >  %  >  such  that 
T  <  <  tx  <  s2  <  £2  <  •  •  •  <  SR  <  tK  and  such  that 

(340)  G(sk  ,  tk )  >  r'(tk  —  sk)  +  f  +  e  for  £  ~  1,  2,  ...  ,  K 

Summing  (340)  over  k  yields: 

(341)  £  G{sk  ,  tk)  >  r-  £  (/*  -  5*)  +  K-f  +  ff-e 

*-i  *-i 

Now  apply  assumption  (335)  to  G{tx  ,  s2)  ,  G(t2  ,  s3)  ,  ...  ,  G(tK_x  »  %)  anc^ 
sum  over  k. 

K- l  K- l r  1 

E  G{tk,sk+l)  >  £  -  h)  ~  f'\ 

k- 1  ±-  1  L  J 

(342)  =  r-A £(5i+1  -  tk)  ~  (K  -  1  )*/' 

*- 1 

(Note  that  (342)  holds  even  if  A'  =  1  .)  Definition  (334)  can  be  used  to  add 
(341)  and  (342)  together. 
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£(si  *  Ik) 


tK 

E  *(«) 

«— «i+ 1 


K  U  AT—  l  4*+i 

E  E  ?(“)  +  E  E  ?(u) 

it—  1  U—  S*+  1  fc— 1  tf— f*+l 


/T  /T—  1 

E  G{sk ,  tk)  +  2  ^(*jt » s*+i) 

*-i  *- i 


>  r*  E  (<*  -  sk)  +  K'f'  +  K-e 

k-  1 


+  r-E  (•*+!"  «*)  -(*"  !)•/' 

*-i 


(343) 


r'(tK  -  *x)  +  AT'e  +  /' 


For  sufficiently  large  K,  viz.: 


(344) 


K  > 


r  -  r 


relation  (343)  contradicts  assumption  (335).  This  completes  the  proof  of 


Lemma  8. 


A.2  Lemma  9:  Smoothness  of  a  Bernoulli  Process 

Let  g(t)  be  a  Bernoulli  process  with  rate  n  ,  0  <  n  <  1  .  Let  G(s,  t)  be 
the  number  of  successes  among  ?(s  +  l),  ff(s+2),  ...  ,  g(t)  ;  note  that 
G(s,  t)  —  0  if  s  >  t  .  For  any  positive  real  number  A,  any  integer  T,  and 
any  positive  integer  r ,  it  follows  that 

[  G(s,  t)  -  n‘(t  -  s)|  <  A 
for  all  s,  t  such  that  T  <  s  <  t  <  T+r 

Proof  of  Lemma  9 

Let  s  and  t  be  any  integers  such  that  T  <  s  <  t  <  T+t  .  Note  that 
(346)  G(s,  t )  -  G(T,  t)  -  G(7,  s) 

Consequently,  if 

|c(r,s)  -„•(«-  r)|  <  f 

and 

|c(r,i)  -  /x-(<  -  T) |  <  A 

then 

|  G(s,  f)  —  n'{t  —  «)|  <  A 

Therefore,  to  prove  (345)  it  suffices  to  show  that 


A 


(347) 


PROB 


|G(T,U)  -!»•(«-  T)  |  <  f 
for  all  u  such  that  T  <  u  <  T+r 


>  1  - 


This  is  the  same  as  proving  that 


(348)  PROB 


I  £  c(r+*-i,  r+Jt)  -  /x-z^i  < 

k-  1 

for  all  K  such  that  1  <  K  <  r 


r 


A2 


Inequality  (348)  follows  from  Kolmogorov’s  inequality  [3]  and  the  fact  that 
VARIANCE  {  G{T,  T+t)  }  -  /r(l -#i)-r 

(349)  <  j 


This  completes  the  proof  of  Lemma  9. 
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A. 3  Lemma  10:  A  Corollary  of  the  Strong  Law  of  Large  Numbers 

Suppose  that  g^,  go,  ...  are  independent,  identically  distributed  random 
variables  with  mean  fz.  For  all  integers  K  and  g  such  that  0  <  K  <  g  , 
define  G q  and  Gq  as  follows: 

(350)  Gf  =  -  •  £  9k 

9  k- 1 

(351)  G.  —  max  G f 

0<  K<  q  9 

It  follows  that,  with  probability  one, 

(352)  Gq  y  max(0,  n] 

oo 

Proof  of  Lemma  10 

It  follows  from  (350)  and  the  strong  law  of  large  numbers  [3]  that,  with 
probability  one, 


(353) 


00 


This  means  that  there  exists  a  subset  T  of  the  sample  space  such  that 
(35-t)  PROP  {  T  }  =  1 

and  such  that,  for  every  sample  point  7  in  T  and  every  positive  real  number  e  , 
there  exists  a  positive  integer  Q( 7,  e)  such  that 


I 

I 

' 

! 

i 

j  (355)  |  GiJb)  -  It  I  <  e  for  all  K  >  Q(i,  e) 

I 

;  To  prove  (352),  it  will  be  shown  that  for  every  7  in  T  and  every  positive  real 

I 

j  number  e  ,  there  exists  a  positive  integer  Q'( 7,  e)  such  that 

|  Gq (7)  -  max[0,  n\  |  <  e  for  all  q  >  Q'{ 7,  e) 

Let  7  be  any  sample  point  in  T  and  let  e  be  any  positive  real  number.  Define 
|  Q'(l,  e)  as  follows. 


Q\ 7,  f)  —  max 

' 

1  Qh.  * ) 

' 

(356) 

Qin* «)» 

E  \oM\ 

e  k- 1 

Let  q  be  any  integer  such  that 

(357)  q  >  Q\ 7,  e) 

The  goal  is  to  show  that 

(358)  |  Gq{ 7)  -  max[0,  n\  |  <  e 


First,  let  us  find  a  lower  bound  for  Gq{ 7)  .  It  follows  from  (351)  and  (350) 


Using  (351),  (355),  (357),  and  (356),  a  different  lower  bound  can  be  found: 


GJ~i)  -  max  C5(- y) 

*  o <K<q  1 

>  Gfr) 

(360)  >  n  -  € 

Combining  (359)  and  (360)  yields: 

(361)  Gj('y)  >  max[0,  p\  -  e 

This  is  half  of  the  desired  result  (358). 

Now  an  upper  bound  can  be  found  for  Gg(~ y)  by  proving  the  fcilow'm 
bound  for  G^(~ y)  . 

(362)  Gq'(~‘)  <  max[0.  n\  +  t  for  K  =  0.  1 . q 

The  small  and  large  values  of  K  must  be  treated  separately.  Fo 
0  <  K  <  Q(7,  e)  ,  apply  (350),  (357),  and  (356): 


9  k-i 


<  -  •  S  UtHI 

?  i-1 


<  1 


<?(7,  I  ,  M 

E  l**(Tf)l 

k-  1 


< 


1 

QH  e) 


Q(r  0 

E 


t- 1 


\9k{l)\ 


<  £ 


<  max[0,  /*]  +  £ 


For  Q(7,  f)  <  K  <  ?  ,  apply  (350)  and  (355): 


=  —  *  gRi) 

<  —*(/*  +  €) 

9 

<  —  •  (max[0,  n]  -F  e) 

9 

<  max[0,  n)  +  e 

This  completes  the  proof  of  (362).  From  (351)  and  (362),  it  follows  that 

G'w  “  c*b) 

(363)  <  max[0,  /u]  +  e 

This  is  the  second  half  of  the  desired  result  (358).  completing  the  proof  of 


Lemma  10. 
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GLOSSARY  OF  NOTATION 


SYMBOL  MEANING  DEFINED  IN 


[s,  tj 

Time  interval  from  beginning  of  slot  5 
to  end  of  slot  t ;  null  if  s  >  t 

§2.1 

(s,  t ] 

Same  as  [s+1,  t ] 

§2.1 

[s,  t ) 

Same  as  [s,  i — 1] 

§2.1 

(s,  0 

Same  as  [s  +  1,  £—1] 

§2.1 

S 

Number  of  sessions  in  network 

§2.2 

N'{1) 

Number  of  sessions  using  link  l 

§2.2 

N(x,  h) 

Number  of  sessions  using  hop  h  of 
session  x 

§2.2 

N{x) 

Maximum  number  of  sessions  using  any 
link  in  path  of  session  x 

§2.2 

N 

Maximum  number  of  sessions  sharing 
any  link  in  network 

§  2.2 

H(x)  Number  of  links  in  path  of  session  x 


H 


Maximum  number  of  links  in  path  of 
any  session  in  network 


§2.2 

§2.2 


i 

W{ x,  A) 


W' 


I 

:  w" 


B(x,  h,  t ) 


TSS 


m(x,  A) 


M{x ,  h ) 


P{x,  h,  s,  t) 
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Window  size  (capacity)  for  buffer  h  of 
session  x 

Maximum  window  size  for  any  session  x 
at  any  buffer  h  in  the  range 

1  <  h  <  H(x) 

Maximum  window  size  for  any  session  x 
at  any  buffer  h  in  the  range 

2  <h<  H(x)+ 1 


Level  of  buffer  h  of  session  x  at  time  t 


Time  when  throughputs  and  buffer 
levels  stabilize 

Minimum  level  of  buffer  h  of  session  x 
after  time  Tss 

Maximum  level  of  buffer  h  of  session  x 
after  time  TSs 


Throughput  (number  of  transmitted 
packets)  for  session  x  over  hop  h  during 
interval  (s,  f] 


P'{x,  l,  s ,  t ) 


Throughput  (number  of  transmitted 
packets)  for  session  x  over  link  l  during 
interval  (s,  £] 


Ra(*) 


Long-term  average  throughput  of 
session  x 


§2.2 


T(x,  h,  p) 

Time  slot  during  which  session  x 
transmits  packet  p  over  hop  k 

§2.2 

B(x,  p) 

Cross-network  delay  for  packet  p  of 
session  x 

§2.2 

C(x,  h ,  s,  t) 

Number  of  chances  for  session  x  to 
transmit  over  hop  h  during  interval 
(s,  t];  C(x,  0,  s,  t)  is  number  of 
attempts  by  x  to  insert  packets  into 
buffer  1  during  (s,  f] 

§  2.4.1;  § 

C'(x,  l,  s,  t) 

Number  of  chances  for  session  x  to 
transmit  over  link  /  during  interval 
(s,  t] 

§  2.4.1 

A'(l) 

Schedule  delay  bound  for  link  l 

§  2.4.3 

A(x,  h) 

Schedule  delay  bound  for  hop  h  of 
session  x 

§  2.4.3 

A{x) 

Maximum  schedule  delay  bound  for  any 
link  in  path  of  session  x 

§  2.4.3 

n 

Sample  space  consisting  of  all  possible 
sample  paths  of  system  demand 

§  2.5 

X(x) 

Demand  rate  of  session  x 

§2.5 
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Rf(  0 


ith  smallest  distinct  fair  rate;  fair  rate  §  4.1 
for  any  session  with  congestion  index  i 


I(x )  Congestion  index  of  session  x  §  4.1 

/  Number  of  distinct  fair  rates;  §  4.1 

maximum  congestion  index  of  any 
session 


Rc{x,  h ) 

Lower  bound  on  fair  rate  of  chances  for 
session  x  at  hop  h 

§  4.1 

R'c(x,  0 

Lower  bound  on  fair  rate  of  chances  for 
session  x  at  link  l 

§  4.1 

DCL>  Ecl,  E  cl 

Functions  used  as  lower  bounds  on 
numbers  of  chances  received  by  sessions 

§4.2 

dpl  »  EPL ,  F PL 

Functions  used  as  lower  bounds  on 
numbers  of  packets  transmitted  by 
sessions 

§4.2 

Epiji  Epu ,  FPU,  F"pu 

Functions  used  as  upper  bounds  on 
numbers  of  packets  transmitted  by 
sessions 

§4.2 
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